[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:24 dist-test-slave-lkx2 krb5kdc[12103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:24 dist-test-slave-lkx2 krb5kdc[12103](info): set up 2 sockets
Aug 12 21:08:24 dist-test-slave-lkx2 krb5kdc[12103](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 21:08:26.160188 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:26.285535 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:33269
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:33269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:26.409826 12125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:26.410082 12125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:26.410109 12125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:26.411458 12125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:26.411504 12125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:26.411516 12125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:26.411527 12125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:26.411538 12125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:26.413602 12125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:33269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:33269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:26.413981 12125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:26.414243 12125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:26.421366 12142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:26.421432 12125 server_base.cc:1034] running on GCE node
W20260812 21:08:26.421506 12139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:26.422451 12138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:26.422762 12125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:26.423206 12125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:26.424399 12125 hybrid_clock.cc:648] HybridClock initialized: now 1786568906424362 us; error 84 us; skew 500 ppm
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:26.427937 12125 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:26.428813 12125 webserver.cc:460] Webserver started at http://127.11.207.190:41311/ using document root <none> and password file <none>
I20260812 21:08:26.429147 12125 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:26.429216 12125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:26.429412 12125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:26.430859 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "8525ed2cb4c74416b91574e00aac7154"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "046847f5b324bf89287951e4989bcb5b"
server_key_iv: "cd0103e6bc04a96a83541b143f844ccc"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.431365 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "8525ed2cb4c74416b91574e00aac7154"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "046847f5b324bf89287951e4989bcb5b"
server_key_iv: "cd0103e6bc04a96a83541b143f844ccc"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.433686 12125 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260812 21:08:26.435006 12168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.435344 12125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:26.435432 12125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "8525ed2cb4c74416b91574e00aac7154"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "046847f5b324bf89287951e4989bcb5b"
server_key_iv: "cd0103e6bc04a96a83541b143f844ccc"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.435524 12125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:26.447781 12125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:26.449887 12125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:26.450034 12125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:26.462599 12125 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:33269
I20260812 21:08:26.462744 12266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:33269 every 8 connection(s)
I20260812 21:08:26.463379 12125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:26.465076 12267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:26.468545 12267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154: Bootstrap starting.
I20260812 21:08:26.470046 12267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:26.470978 12267 log.cc:824] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:26.472183 12267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154: No bootstrap required, opened a new log
I20260812 21:08:26.472705 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12125
I20260812 21:08:26.472790 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:26.473579 12094 external_mini_cluster.cc:1428] Setting key 2e426ddf990e95a302537bceb2b1e171
I20260812 21:08:26.477100 12267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8525ed2cb4c74416b91574e00aac7154" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33269 } }
I20260812 21:08:26.477329 12267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:26.477438 12267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8525ed2cb4c74416b91574e00aac7154, State: Initialized, Role: FOLLOWER
I20260812 21:08:26.477640 12267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8525ed2cb4c74416b91574e00aac7154" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33269 } }
I20260812 21:08:26.477810 12267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:26.477859 12267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:26.477901 12267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:26.479324 12267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8525ed2cb4c74416b91574e00aac7154" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33269 } }
I20260812 21:08:26.479578 12267 leader_election.cc:302] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8525ed2cb4c74416b91574e00aac7154; no voters:
I20260812 21:08:26.479816 12267 leader_election.cc:288] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:26.479871 12283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:26.480142 12283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [term 1 LEADER]: Becoming Leader. State: Replica: 8525ed2cb4c74416b91574e00aac7154, State: Running, Role: LEADER
I20260812 21:08:26.480289 12283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8525ed2cb4c74416b91574e00aac7154" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33269 } }
I20260812 21:08:26.480432 12267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:26.481014 12284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8525ed2cb4c74416b91574e00aac7154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8525ed2cb4c74416b91574e00aac7154" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33269 } } }
I20260812 21:08:26.481106 12284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:26.481564 12285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8525ed2cb4c74416b91574e00aac7154. Latest consensus state: current_term: 1 leader_uuid: "8525ed2cb4c74416b91574e00aac7154" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8525ed2cb4c74416b91574e00aac7154" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33269 } } }
I20260812 21:08:26.481648 12285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:26.483129 12289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:26.484794 12289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:26.487617 12289 catalog_manager.cc:1347] Generated new cluster ID: 23e24c1e94534a2a85860089759af3e2
I20260812 21:08:26.487675 12289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:26.495229 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:26.474848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.11.207.190:33269)
0812 21:08:26.475447 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:26.475458 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:26.475626 (+ 168us) server_negotiation.cc:407] Connection header received
0812 21:08:26.478966 (+ 3340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:26.478974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:26.479094 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:26.479269 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:26.481823 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.484642 (+ 2819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.486014 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.486604 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.486710 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:26.490939 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:26.490943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:26.490944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:26.490955 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:26.493111 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:26.493542 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:26.493545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:26.493545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:26.493709 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:26.493964 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:26.493965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:26.493965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:26.494217 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:26.494581 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:26.494673 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 21:08:26.494856 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":252,"threads_started":1}
I20260812 21:08:26.512323 12289 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:26.512898 12289 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:26.517678 12289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8525ed2cb4c74416b91574e00aac7154: Generated new TSK 0
I20260812 21:08:26.517867 12289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:26.559369 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33269
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:26.695173 12322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:26.695389 12322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:26.695415 12322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:26.696633 12322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:26.696683 12322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:26.696712 12322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:26.698279 12322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33269
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:26.698608 12322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:26.698876 12322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:26.705050 12336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:26.705125 12339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:26.705070 12332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:26.705832 12322 server_base.cc:1034] running on GCE node
I20260812 21:08:26.706075 12322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:26.706351 12322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:26.707544 12322 hybrid_clock.cc:648] HybridClock initialized: now 1786568906707528 us; error 28 us; skew 500 ppm
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:26.711727 12322 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:26.712402 12322 webserver.cc:460] Webserver started at http://127.11.207.129:43403/ using document root <none> and password file <none>
I20260812 21:08:26.712718 12322 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:26.712792 12322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:26.712957 12322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:26.714915 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "d6e785bdca7a400791336fd563e281c2"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "d911a737322750f13ca1b72a4103d953"
server_key_iv: "0370adfe8d0289828e363fa809617ae5"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.715502 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "d6e785bdca7a400791336fd563e281c2"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "d911a737322750f13ca1b72a4103d953"
server_key_iv: "0370adfe8d0289828e363fa809617ae5"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.717414 12322 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260812 21:08:26.718276 12350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.718444 12322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:08:26.718539 12322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "d6e785bdca7a400791336fd563e281c2"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "d911a737322750f13ca1b72a4103d953"
server_key_iv: "0370adfe8d0289828e363fa809617ae5"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.718613 12322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:26.736954 12322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:26.738952 12322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:26.739074 12322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:26.739509 12322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:26.739910 12322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:26.739995 12322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.740024 12322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:26.740041 12322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.752652 12322 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:36219
I20260812 21:08:26.752696 12535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:36219 every 8 connection(s)
I20260812 21:08:26.753046 12322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:26.757828 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12322
I20260812 21:08:26.757910 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:26.758041 12094 external_mini_cluster.cc:1428] Setting key f33b8d1d180d7adb168b9d006b29f379
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:26.763826 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:26.754239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:39695 (local address 127.11.207.190:33269)
0812 21:08:26.754290 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:26.754292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:26.754690 (+ 398us) server_negotiation.cc:407] Connection header received
0812 21:08:26.755260 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:26.755261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:26.755276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:26.755318 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:26.756481 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.757024 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.758002 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.758268 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.758339 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:26.760967 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:26.760969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:26.760970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:26.760979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:26.762405 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:26.762804 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:26.762806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:26.762806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:26.762877 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:26.763102 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:26.763103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:26.763103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:26.763242 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:26.763308 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:26.763599 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 21:08:26.763705 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:08:26.763851 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:26.754245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:33269 (local address 127.11.207.129:39695)
0812 21:08:26.754525 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:26.754575 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:26.755160 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:26.755383 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:26.755386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:26.756126 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:26.756390 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:26.756397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.757092 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.757092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:26.757929 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:26.757933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.758347 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.758348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:26.758513 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:26.759113 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:26.759122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:26.760857 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:26.762494 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:26.762501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:26.762503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:26.762731 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:26.762940 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:26.762941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:26.762941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:26.763042 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:26.763299 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:26.763302 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:26.763463 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 21:08:26.763545 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:08:26.763674 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":154,"threads_started":1}
I20260812 21:08:26.764591 12536 heartbeater.cc:347] Connected to a master server at 127.11.207.190:33269
I20260812 21:08:26.764693 12536 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:26.764868 12536 heartbeater.cc:511] Master 127.11.207.190:33269 requested a full tablet report, sending...
I20260812 21:08:26.765821 12201 ts_manager.cc:194] Registered new tserver with Master: d6e785bdca7a400791336fd563e281c2 (127.11.207.129:36219)
I20260812 21:08:26.766546 12201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:39695
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:26.787562 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33269
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:26.901899 12545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:26.902110 12545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:26.902143 12545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:26.903321 12545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:26.903414 12545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:26.903502 12545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:26.905262 12545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33269
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:26.905526 12545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:26.905745 12545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:26.911209 12552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:26.911283 12556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:26.911221 12560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:26.911531 12545 server_base.cc:1034] running on GCE node
I20260812 21:08:26.911722 12545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:26.912019 12545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:26.913151 12545 hybrid_clock.cc:648] HybridClock initialized: now 1786568906913144 us; error 31 us; skew 500 ppm
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:26.915580 12545 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:26.916172 12545 webserver.cc:460] Webserver started at http://127.11.207.130:45725/ using document root <none> and password file <none>
I20260812 21:08:26.916440 12545 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:26.916493 12545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:26.916666 12545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:26.917719 12545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "abd6ca6bfa624b8a97fb0f146f462f21"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "0ee27b54f363e0bb965495d0797f2332"
server_key_iv: "0eaa8a8b081b735ba60a1d04080ec400"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.918054 12545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "abd6ca6bfa624b8a97fb0f146f462f21"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "0ee27b54f363e0bb965495d0797f2332"
server_key_iv: "0eaa8a8b081b735ba60a1d04080ec400"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.919468 12545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:26.920171 12573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.920392 12545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:26.920485 12545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "abd6ca6bfa624b8a97fb0f146f462f21"
format_stamp: "Formatted at 2026-08-12 21:08:26 on dist-test-slave-lkx2"
server_key: "0ee27b54f363e0bb965495d0797f2332"
server_key_iv: "0eaa8a8b081b735ba60a1d04080ec400"
server_key_version: "encryptionkey@0"
I20260812 21:08:26.920557 12545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:26.943912 12545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:26.945780 12545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:26.945921 12545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:26.946143 12545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:26.946528 12545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:26.946565 12545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.946596 12545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:26.946620 12545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:26.956211 12545 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:33665
I20260812 21:08:26.956257 12820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:33665 every 8 connection(s)
I20260812 21:08:26.956669 12545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:08:26 dist-test-slave-lkx2 krb5kdc[12103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:26.963959 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12545
I20260812 21:08:26.964039 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:26.964188 12094 external_mini_cluster.cc:1428] Setting key 24c8517ed949ca91bc7ebffa53550918
I20260812 21:08:26.965572 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:26.957447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:44791 (local address 127.11.207.190:33269)
0812 21:08:26.957506 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:26.957509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:26.957746 (+ 237us) server_negotiation.cc:407] Connection header received
0812 21:08:26.958175 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:26.958176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:26.958188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:26.958243 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:26.959251 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.959783 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.960735 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.960926 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.960977 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:26.962980 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:26.962982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:26.962982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:26.962990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:26.964195 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:26.964658 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:26.964659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:26.964660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:26.964725 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:26.964949 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:26.964950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:26.964950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:26.965065 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:26.965121 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:26.965399 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 21:08:26.965483 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:08:26.965798 12824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:26.957409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:33269 (local address 127.11.207.130:44791)
0812 21:08:26.957574 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:26.957620 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:26.958093 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:26.958305 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:26.958309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:26.958962 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:26.959162 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:26.959168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.959850 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.959851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:26.960668 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:26.960670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:26.960983 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:26.960984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:26.961079 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:26.961563 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:26.961572 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:26.962877 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:26.964272 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:26.964280 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:26.964282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:26.964586 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:26.964781 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:26.964781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:26.964782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:26.964892 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:26.965121 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:26.965129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:26.965278 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 21:08:26.965368 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:08:26.965525 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260812 21:08:26.966298 12822 heartbeater.cc:347] Connected to a master server at 127.11.207.190:33269
I20260812 21:08:26.966413 12822 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:26.966601 12822 heartbeater.cc:511] Master 127.11.207.190:33269 requested a full tablet report, sending...
I20260812 21:08:26.967010 12201 ts_manager.cc:194] Registered new tserver with Master: abd6ca6bfa624b8a97fb0f146f462f21 (127.11.207.130:33665)
I20260812 21:08:26.967371 12201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:44791
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:26.992121 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33269
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:27.078578 12831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:27.078800 12831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:27.078826 12831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:27.080000 12831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:27.080042 12831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:27.080068 12831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:27.081617 12831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33269
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:27.081869 12831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:27.082098 12831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:27.087277 12836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:27.087342 12831 server_base.cc:1034] running on GCE node
W20260812 21:08:27.087277 12837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:27.087311 12839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:27.087662 12831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:27.087971 12831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:27.089113 12831 hybrid_clock.cc:648] HybridClock initialized: now 1786568907089102 us; error 28 us; skew 500 ppm
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568907, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:27.091509 12831 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:27.092028 12831 webserver.cc:460] Webserver started at http://127.11.207.131:44165/ using document root <none> and password file <none>
I20260812 21:08:27.092211 12831 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:27.092250 12831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:27.092365 12831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:27.093482 12831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "d74a3f59f8c749e79f553b330cda87b6"
format_stamp: "Formatted at 2026-08-12 21:08:27 on dist-test-slave-lkx2"
server_key: "26b8fbd96404dd92692c370b68c39e10"
server_key_iv: "7568b01679f72c7f8e8d02334f9fdbbe"
server_key_version: "encryptionkey@0"
I20260812 21:08:27.093858 12831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "d74a3f59f8c749e79f553b330cda87b6"
format_stamp: "Formatted at 2026-08-12 21:08:27 on dist-test-slave-lkx2"
server_key: "26b8fbd96404dd92692c370b68c39e10"
server_key_iv: "7568b01679f72c7f8e8d02334f9fdbbe"
server_key_version: "encryptionkey@0"
I20260812 21:08:27.095088 12831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:27.095762 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:27.095928 12831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:27.096005 12831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "d74a3f59f8c749e79f553b330cda87b6"
format_stamp: "Formatted at 2026-08-12 21:08:27 on dist-test-slave-lkx2"
server_key: "26b8fbd96404dd92692c370b68c39e10"
server_key_iv: "7568b01679f72c7f8e8d02334f9fdbbe"
server_key_version: "encryptionkey@0"
I20260812 21:08:27.096069 12831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:27.106568 12831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:27.108788 12831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:27.108919 12831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:27.109172 12831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:27.109472 12831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:27.109505 12831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:27.109527 12831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:27.109550 12831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:27.125806 12831 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:36577
I20260812 21:08:27.125941 13076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:36577 every 8 connection(s)
I20260812 21:08:27.126216 12831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:27.128129 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12831
I20260812 21:08:27.128214 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:27.128373 12094 external_mini_cluster.cc:1428] Setting key 0c92d1f34e2ef7b843061d2142e9b43a
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[12103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568907, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:27.135661 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:27.127030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:56927 (local address 127.11.207.190:33269)
0812 21:08:27.127092 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:27.127094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:27.127303 (+ 209us) server_negotiation.cc:407] Connection header received
0812 21:08:27.127795 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:27.127796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:27.127809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:27.127861 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:27.128911 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.129565 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.130542 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.130748 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.130817 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:27.133100 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:27.133102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:27.133102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:27.133109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:27.134390 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:27.134760 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:27.134762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:27.134762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:27.134816 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:27.135044 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:27.135045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:27.135045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:27.135161 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:27.135219 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:27.135480 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 21:08:27.135571 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:08:27.135860 13088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:27.126985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:33269 (local address 127.11.207.131:56927)
0812 21:08:27.127195 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:27.127233 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:27.127712 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:27.127921 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:27.127925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:27.128609 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:27.128817 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:27.128824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.129650 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.129651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:27.130454 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:27.130457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.130827 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.130828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:27.130954 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:27.131453 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:27.131461 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:27.133008 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:27.134465 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:27.134473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:27.134475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:27.134695 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:27.134870 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:27.134870 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:27.134871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:27.134976 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:27.135216 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:27.135224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:27.135362 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 21:08:27.135449 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:08:27.135601 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":143,"threads_started":1}
I20260812 21:08:27.136268 13083 heartbeater.cc:347] Connected to a master server at 127.11.207.190:33269
I20260812 21:08:27.136372 13083 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:27.136550 13083 heartbeater.cc:511] Master 127.11.207.190:33269 requested a full tablet report, sending...
I20260812 21:08:27.136920 12201 ts_manager.cc:194] Registered new tserver with Master: d74a3f59f8c749e79f553b330cda87b6 (127.11.207.131:36577)
I20260812 21:08:27.137271 12201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:56927
I20260812 21:08:27.140313 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568907, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[12103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568907, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:27.166013 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:27.158241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.11.207.129:36219)
0812 21:08:27.158456 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:27.158462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:27.158475 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:08:27.158524 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:27.158526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:27.158542 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:27.158603 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:27.159164 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.159913 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.160668 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.161019 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.161080 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:27.163320 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:27.163330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:27.163330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:27.163337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:27.165031 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:27.165312 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:27.165314 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:27.165315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:27.165411 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:27.165541 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:27.165541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:27.165542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:27.165669 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:27.165803 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:27.165840 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 21:08:27.165906 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
W20260812 21:08:27.166565 12396 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33106
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[12103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[12103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:27.190668 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:27.184042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.11.207.129:36219)
0812 21:08:27.184118 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:27.184120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:27.184132 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:08:27.184256 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:27.184257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:27.184270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:27.184331 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:27.184968 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.185469 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.186181 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:27.186396 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:27.186450 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:27.188645 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:27.188646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:27.188646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:27.188653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:27.189747 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:27.190026 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:27.190027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:27.190027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:27.190072 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:27.190282 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:27.190283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:27.190283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:27.190386 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:27.190480 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:27.190537 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:08:27.190597 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:08:27.191220 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12322
I20260812 21:08:27.203712 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12545
I20260812 21:08:27.216457 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12831
I20260812 21:08:27.237907 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12125
2026-08-12T21:08:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2439 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[13119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[13119](info): set up 2 sockets
Aug 12 21:08:27 dist-test-slave-lkx2 krb5kdc[13119](info): commencing operation
krb5kdc: starting...
W20260812 21:08:28.455333 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:28 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:28.575280 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:42077
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:42077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:28.657179 13146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:28.657390 13146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:28.657413 13146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:28.658553 13146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:28.658594 13146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:28.658607 13146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:28.658629 13146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:28.658651 13146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:28.660120 13146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:42077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:42077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:28.660382 13146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:28.660599 13146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:28.665688 13155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:28.666025 13157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:28.665688 13159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:28.667196 13146 server_base.cc:1034] running on GCE node
I20260812 21:08:28.667471 13146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:28.667764 13146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:28.668905 13146 hybrid_clock.cc:648] HybridClock initialized: now 1786568908668891 us; error 28 us; skew 500 ppm
Aug 12 21:08:28 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:28.671406 13146 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:28.671814 13146 webserver.cc:460] Webserver started at http://127.11.207.190:37129/ using document root <none> and password file <none>
I20260812 21:08:28.672017 13146 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:28.672058 13146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:28.672148 13146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:28.673305 13146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "02161c1b764047c3b81ed25f48942b09"
format_stamp: "Formatted at 2026-08-12 21:08:28 on dist-test-slave-lkx2"
server_key: "bacbeae593e9deef15002739742a486d"
server_key_iv: "f6eebc168d3e088b21836005e3172f42"
server_key_version: "encryptionkey@0"
I20260812 21:08:28.673625 13146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "02161c1b764047c3b81ed25f48942b09"
format_stamp: "Formatted at 2026-08-12 21:08:28 on dist-test-slave-lkx2"
server_key: "bacbeae593e9deef15002739742a486d"
server_key_iv: "f6eebc168d3e088b21836005e3172f42"
server_key_version: "encryptionkey@0"
I20260812 21:08:28.674916 13146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:28.675602 13172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:28.675761 13146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:28.675850 13146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "02161c1b764047c3b81ed25f48942b09"
format_stamp: "Formatted at 2026-08-12 21:08:28 on dist-test-slave-lkx2"
server_key: "bacbeae593e9deef15002739742a486d"
server_key_iv: "f6eebc168d3e088b21836005e3172f42"
server_key_version: "encryptionkey@0"
I20260812 21:08:28.675907 13146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:28.684716 13146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:28.686756 13146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:28.686869 13146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:28.691064 13146 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:42077
I20260812 21:08:28.691099 13259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:42077 every 8 connection(s)
I20260812 21:08:28.691483 13146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:28.692322 13261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:28.694722 13261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09: Bootstrap starting.
I20260812 21:08:28.695245 13261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:28.695497 13261 log.cc:824] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:28.696182 13261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09: No bootstrap required, opened a new log
I20260812 21:08:28.697777 13261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02161c1b764047c3b81ed25f48942b09" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 42077 } }
I20260812 21:08:28.697912 13261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:28.697934 13261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02161c1b764047c3b81ed25f48942b09, State: Initialized, Role: FOLLOWER
I20260812 21:08:28.698040 13261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02161c1b764047c3b81ed25f48942b09" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 42077 } }
I20260812 21:08:28.698087 13261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:28.698112 13261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:28.698141 13261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:28.698598 13261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02161c1b764047c3b81ed25f48942b09" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 42077 } }
I20260812 21:08:28.698680 13261 leader_election.cc:302] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02161c1b764047c3b81ed25f48942b09; no voters:
I20260812 21:08:28.698805 13261 leader_election.cc:288] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:28.698848 13266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:28.699018 13261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:28.699031 13266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [term 1 LEADER]: Becoming Leader. State: Replica: 02161c1b764047c3b81ed25f48942b09, State: Running, Role: LEADER
I20260812 21:08:28.699139 13266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02161c1b764047c3b81ed25f48942b09" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 42077 } }
I20260812 21:08:28.699523 13269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02161c1b764047c3b81ed25f48942b09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02161c1b764047c3b81ed25f48942b09" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 42077 } } }
I20260812 21:08:28.699611 13269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:28.699668 13272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02161c1b764047c3b81ed25f48942b09. Latest consensus state: current_term: 1 leader_uuid: "02161c1b764047c3b81ed25f48942b09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02161c1b764047c3b81ed25f48942b09" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 42077 } } }
I20260812 21:08:28.699743 13272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:28.700098 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13146
I20260812 21:08:28.700182 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:28.700374 12094 external_mini_cluster.cc:1428] Setting key 90e1c0cfb9c3f4c53f2a0d135e006247
I20260812 21:08:28.700625 13288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:28.701061 13288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:28.702723 13288 catalog_manager.cc:1347] Generated new cluster ID: b1f089c329c44e29a76f86dc56afa6d6
I20260812 21:08:28.702780 13288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:28 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:28.711546 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:28.700830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.11.207.190:42077)
0812 21:08:28.701003 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:28.701013 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:28.701175 (+ 162us) server_negotiation.cc:407] Connection header received
0812 21:08:28.701374 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:28.701382 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:28.701439 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:28.701598 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:28.702287 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:28.703538 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:28.704622 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:28.705219 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:28.705310 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:28.707977 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:28.707982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:28.707982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:28.707995 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:28.709940 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:28.710295 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:28.710298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:28.710298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:28.710443 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:28.710614 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:28.710614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:28.710615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:28.710927 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:28.711098 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:28.711166 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 21:08:28.711306 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260812 21:08:28.715631 13288 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:28.716122 13288 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:28.721705 13288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02161c1b764047c3b81ed25f48942b09: Generated new TSK 0
I20260812 21:08:28.721894 13288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:28.753409 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:42077
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:28.847707 13314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:28.847934 13314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:28.847958 13314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:28.849164 13314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:28.849211 13314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:28.849237 13314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:28.850781 13314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:42077
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:28.850991 13314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:28.851179 13314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:28.856045 13323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:28.856033 13327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:28.856033 13325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:28.856330 13314 server_base.cc:1034] running on GCE node
I20260812 21:08:28.856593 13314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:28.856796 13314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:28.857931 13314 hybrid_clock.cc:648] HybridClock initialized: now 1786568908857919 us; error 29 us; skew 500 ppm
Aug 12 21:08:28 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:28.860510 13314 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:28.860950 13314 webserver.cc:460] Webserver started at http://127.11.207.129:39267/ using document root <none> and password file <none>
I20260812 21:08:28.861141 13314 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:28.861179 13314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:28.861269 13314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:28.862324 13314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "67d0335e843b468f9bb678115ab98eba"
format_stamp: "Formatted at 2026-08-12 21:08:28 on dist-test-slave-lkx2"
server_key: "e842390eba706a89d3b190bde01bcc2b"
server_key_iv: "30c053bd09b41d99564af8666bd450ed"
server_key_version: "encryptionkey@0"
I20260812 21:08:28.862627 13314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "67d0335e843b468f9bb678115ab98eba"
format_stamp: "Formatted at 2026-08-12 21:08:28 on dist-test-slave-lkx2"
server_key: "e842390eba706a89d3b190bde01bcc2b"
server_key_iv: "30c053bd09b41d99564af8666bd450ed"
server_key_version: "encryptionkey@0"
I20260812 21:08:28.863835 13314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:08:28.864493 13344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:28.864648 13314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:28.864740 13314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "67d0335e843b468f9bb678115ab98eba"
format_stamp: "Formatted at 2026-08-12 21:08:28 on dist-test-slave-lkx2"
server_key: "e842390eba706a89d3b190bde01bcc2b"
server_key_iv: "30c053bd09b41d99564af8666bd450ed"
server_key_version: "encryptionkey@0"
I20260812 21:08:28.864807 13314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:28.882594 13314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:28.884884 13314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:28.885027 13314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:28.885279 13314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:28.885622 13314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:28.885664 13314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:28.885689 13314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:28.885715 13314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:28.891714 13314 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:34793
I20260812 21:08:28.891762 13557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:34793 every 8 connection(s)
I20260812 21:08:28.892078 13314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:08:28 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568908, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:28.899798 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13314
I20260812 21:08:28.899911 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:28.900113 12094 external_mini_cluster.cc:1428] Setting key c2681324905a40a3f99bba97ca31e601
I20260812 21:08:28.901937 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:28.892895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:55269 (local address 127.11.207.190:42077)
0812 21:08:28.892953 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:28.892955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:28.893151 (+ 196us) server_negotiation.cc:407] Connection header received
0812 21:08:28.894361 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:28.894362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:28.894380 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:28.894434 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:28.895115 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:28.895660 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:28.896719 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:28.896922 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:28.896980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:28.899050 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:28.899052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:28.899052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:28.899061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:28.900635 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:28.901094 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:28.901095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:28.901096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:28.901161 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:28.901358 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:28.901359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:28.901359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:28.901488 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:28.901551 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:28.901769 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 21:08:28.901854 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:08:28.902068 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:28.892856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:42077 (local address 127.11.207.129:55269)
0812 21:08:28.893031 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:28.893070 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:28.894234 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:28.894524 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:28.894529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:28.894805 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:28.895016 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:28.895020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:28.895751 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:28.895752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:28.896630 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:28.896632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:28.896996 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:28.896997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:28.897127 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:28.897566 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:28.897575 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:28.898959 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:28.900704 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:28.900713 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:28.900716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:28.901015 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:28.901215 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:28.901215 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:28.901215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:28.901307 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:28.901541 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:28.901544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:28.901657 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 21:08:28.901738 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 21:08:28.901876 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260812 21:08:28.902952 13561 heartbeater.cc:347] Connected to a master server at 127.11.207.190:42077
I20260812 21:08:28.903064 13561 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:28.903262 13561 heartbeater.cc:511] Master 127.11.207.190:42077 requested a full tablet report, sending...
I20260812 21:08:28.903760 13203 ts_manager.cc:194] Registered new tserver with Master: 67d0335e843b468f9bb678115ab98eba (127.11.207.129:34793)
I20260812 21:08:28.904398 13203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:55269
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:28.927138 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:42077
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:29.009860 13570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:29.010083 13570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:29.010116 13570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:29.011320 13570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:29.011364 13570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:29.011401 13570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:29.012965 13570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:42077
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:29.013216 13570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:29.013463 13570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:29.018617 13582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:29.018731 13586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:29.018862 13570 server_base.cc:1034] running on GCE node
W20260812 21:08:29.018739 13577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:29.019105 13570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:29.019325 13570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:29.020473 13570 hybrid_clock.cc:648] HybridClock initialized: now 1786568909020441 us; error 46 us; skew 500 ppm
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:29.022966 13570 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:29.023432 13570 webserver.cc:460] Webserver started at http://127.11.207.130:36643/ using document root <none> and password file <none>
I20260812 21:08:29.023643 13570 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:29.023682 13570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:29.023777 13570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:29.024844 13570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "5d97325d928d411d9397a6ae3b986ce9"
format_stamp: "Formatted at 2026-08-12 21:08:29 on dist-test-slave-lkx2"
server_key: "17a6fd629601342aef3aef96ffe263ba"
server_key_iv: "75bd40438ef88681cf5d7fe5cc7cbd4d"
server_key_version: "encryptionkey@0"
I20260812 21:08:29.025200 13570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "5d97325d928d411d9397a6ae3b986ce9"
format_stamp: "Formatted at 2026-08-12 21:08:29 on dist-test-slave-lkx2"
server_key: "17a6fd629601342aef3aef96ffe263ba"
server_key_iv: "75bd40438ef88681cf5d7fe5cc7cbd4d"
server_key_version: "encryptionkey@0"
I20260812 21:08:29.026623 13570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:29.027271 13596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:29.027455 13570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:29.027542 13570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "5d97325d928d411d9397a6ae3b986ce9"
format_stamp: "Formatted at 2026-08-12 21:08:29 on dist-test-slave-lkx2"
server_key: "17a6fd629601342aef3aef96ffe263ba"
server_key_iv: "75bd40438ef88681cf5d7fe5cc7cbd4d"
server_key_version: "encryptionkey@0"
I20260812 21:08:29.027601 13570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:29.037043 13570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:29.039136 13570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:29.039254 13570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:29.039448 13570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:29.039724 13570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:29.039757 13570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:29.039777 13570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:29.039793 13570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:29.045655 13570 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:35077
I20260812 21:08:29.045720 13812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:35077 every 8 connection(s)
I20260812 21:08:29.046037 13570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:29.052646 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13570
I20260812 21:08:29.052745 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:29.052927 12094 external_mini_cluster.cc:1428] Setting key 3d8cd748bc2b1e00c510c5bcd5c84990
I20260812 21:08:29.055363 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.046727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:59807 (local address 127.11.207.190:42077)
0812 21:08:29.046776 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.046778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.047003 (+ 225us) server_negotiation.cc:407] Connection header received
0812 21:08:29.047947 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.047949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.047963 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.048016 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:29.048701 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.049204 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.050203 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.050412 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.050470 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.052541 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:29.052543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:29.052543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:29.052554 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:29.054043 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.054432 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.054433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.054433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.054499 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.054733 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.054734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.054735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.054856 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:29.054913 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.055186 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.055272 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:29.055564 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.046710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:42077 (local address 127.11.207.130:59807)
0812 21:08:29.046873 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.046914 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.047820 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.048088 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.048092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.048403 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:29.048606 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.048611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.049290 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.049291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.050113 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.050116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.050484 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.050485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.050606 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.051084 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:29.051092 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:29.052447 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:29.054117 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:29.054125 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:29.054128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:29.054363 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:29.054560 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:29.054562 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:29.054562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:29.054678 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:29.054912 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:29.054920 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:29.055067 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.055157 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.055326 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260812 21:08:29.055984 13813 heartbeater.cc:347] Connected to a master server at 127.11.207.190:42077
I20260812 21:08:29.056080 13813 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:29.056262 13813 heartbeater.cc:511] Master 127.11.207.190:42077 requested a full tablet report, sending...
I20260812 21:08:29.056680 13203 ts_manager.cc:194] Registered new tserver with Master: 5d97325d928d411d9397a6ae3b986ce9 (127.11.207.130:35077)
I20260812 21:08:29.057050 13203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:59807
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:29.078833 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:42077
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:29.161406 13820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:29.161631 13820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:29.161667 13820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:29.162843 13820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:29.162885 13820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:29.162921 13820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:29.164548 13820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:42077
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:29.164814 13820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:29.165046 13820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:29.170050 13828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:29.170081 13835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:29.170081 13832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:29.170245 13820 server_base.cc:1034] running on GCE node
I20260812 21:08:29.170536 13820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:29.170825 13820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:29.171983 13820 hybrid_clock.cc:648] HybridClock initialized: now 1786568909171962 us; error 32 us; skew 500 ppm
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:29.174569 13820 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:29.175024 13820 webserver.cc:460] Webserver started at http://127.11.207.131:36365/ using document root <none> and password file <none>
I20260812 21:08:29.175237 13820 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:29.175290 13820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:29.175418 13820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:29.176635 13820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095"
format_stamp: "Formatted at 2026-08-12 21:08:29 on dist-test-slave-lkx2"
server_key: "649bb47a3584c830372735e081f608e4"
server_key_iv: "69af9105792d743d9ee1d7df2e813cb3"
server_key_version: "encryptionkey@0"
I20260812 21:08:29.176986 13820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095"
format_stamp: "Formatted at 2026-08-12 21:08:29 on dist-test-slave-lkx2"
server_key: "649bb47a3584c830372735e081f608e4"
server_key_iv: "69af9105792d743d9ee1d7df2e813cb3"
server_key_version: "encryptionkey@0"
I20260812 21:08:29.178364 13820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:29.179147 13843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:29.179308 13820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:29.179400 13820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095"
format_stamp: "Formatted at 2026-08-12 21:08:29 on dist-test-slave-lkx2"
server_key: "649bb47a3584c830372735e081f608e4"
server_key_iv: "69af9105792d743d9ee1d7df2e813cb3"
server_key_version: "encryptionkey@0"
I20260812 21:08:29.179468 13820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:29.192633 13820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:29.194805 13820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:29.194936 13820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:29.195173 13820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:29.195495 13820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:29.195536 13820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:29.195567 13820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:29.195590 13820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:29.202134 13820 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:35973
I20260812 21:08:29.202188 14061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:35973 every 8 connection(s)
I20260812 21:08:29.202564 13820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:29.204052 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13820
I20260812 21:08:29.204144 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:29.204298 12094 external_mini_cluster.cc:1428] Setting key 4eb19e501faee21a1d0d1fcaabdc22ce
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:29.211570 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.203276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:34133 (local address 127.11.207.190:42077)
0812 21:08:29.203325 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.203327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.203546 (+ 219us) server_negotiation.cc:407] Connection header received
0812 21:08:29.204612 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.204614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.204627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.204702 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:29.205360 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.205887 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.206880 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.207071 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.207125 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.209224 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:29.209226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:29.209226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:29.209235 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:29.210300 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.210662 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.210663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.210664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.210744 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.210960 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.210961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.210961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.211072 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:29.211129 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.211396 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.211479 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:29.211745 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.203260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:42077 (local address 127.11.207.131:34133)
0812 21:08:29.203422 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.203463 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.204508 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.204775 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.204779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.205048 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:29.205274 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.205279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.205972 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.205973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.206800 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.206803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.207138 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.207139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.207261 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.207693 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:29.207701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:29.209129 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:29.210371 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:29.210379 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:29.210381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:29.210596 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:29.210801 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:29.210801 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:29.210802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:29.210905 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:29.211126 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:29.211133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:29.211276 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.211367 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.211527 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260812 21:08:29.212180 14065 heartbeater.cc:347] Connected to a master server at 127.11.207.190:42077
I20260812 21:08:29.212278 14065 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:29.212466 14065 heartbeater.cc:511] Master 127.11.207.190:42077 requested a full tablet report, sending...
I20260812 21:08:29.212790 13203 ts_manager.cc:194] Registered new tserver with Master: 4bb31ca1b57e4fd0bdf072b6b2cc9095 (127.11.207.131:35973)
I20260812 21:08:29.213146 13203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:34133
I20260812 21:08:29.215790 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:08:29.222914 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.217705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.11.207.190:42077)
0812 21:08:29.217752 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.217753 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.217822 (+ 69us) server_negotiation.cc:407] Connection header received
0812 21:08:29.217879 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.217879 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.217887 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.217935 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:29.218413 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.218892 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.219536 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.219732 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.219781 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.220887 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:29.220888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:29.220888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:29.220894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:29.222033 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.222340 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.222341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.222341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.222391 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.222520 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.222520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.222520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.222651 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:29.222743 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.222792 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.222850 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:29.225648 13203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:29.227537 13203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:29.234479 14083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.230494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:34793 (local address 127.0.0.1:42258)
0812 21:08:29.230853 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.230862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.230910 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.231416 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.231418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.231420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:29.231451 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.231452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.232707 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.232708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.233470 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.233472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.234184 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.234185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.234266 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.234280 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.234351 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.234404 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":85,"threads_started":1}
I20260812 21:08:29.234565 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.230452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:35077 (local address 127.0.0.1:54148)
0812 21:08:29.230630 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.230668 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.230736 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.231051 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.231055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.231057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:29.231124 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.231126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.232185 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.232186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.233459 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.233463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.234374 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.234375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.234427 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.234430 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.234477 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.234512 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":44,"threads_started":1}
I20260812 21:08:29.234711 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.230681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:35973 (local address 127.0.0.1:51374)
0812 21:08:29.230988 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.230998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.231046 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.231205 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.231206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.231208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:29.231264 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.231265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.232580 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.232580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.233628 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.233630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.234467 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.234469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.234533 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.234537 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.234593 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.234639 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":81,"threads_started":1}
I20260812 21:08:29.234738 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.230914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.11.207.131:35973)
0812 21:08:29.231061 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.231066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.231076 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:29.231113 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.231115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.231127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.231196 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:29.231368 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.232482 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.233707 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.234392 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.234448 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.234495 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.234614 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.234663 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260812 21:08:29.234858 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.230634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.11.207.129:34793)
0812 21:08:29.231278 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.231282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.231292 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:29.231329 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.231330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.231344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.231417 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:29.231564 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.232615 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.233574 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.234106 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.234175 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.234235 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.234738 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.234786 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":112,"threads_started":1}
I20260812 21:08:29.234947 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.230579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.11.207.130:35077)
0812 21:08:29.230780 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.230785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.230796 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:29.230839 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.230841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.230856 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.230947 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:29.231196 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.232091 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.233552 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.234283 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.234361 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.234418 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.234827 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.234873 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260812 21:08:29.235641 13933 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea0c8d11e39405cb824f1375ab71b8c (DEFAULT_TABLE table=test-table [id=81d999d094e04c55a3cf54ab5b5f826b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:29.235641 13692 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea0c8d11e39405cb824f1375ab71b8c (DEFAULT_TABLE table=test-table [id=81d999d094e04c55a3cf54ab5b5f826b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:29.236013 13933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea0c8d11e39405cb824f1375ab71b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:29.236013 13692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea0c8d11e39405cb824f1375ab71b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:29.237437 13430 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea0c8d11e39405cb824f1375ab71b8c (DEFAULT_TABLE table=test-table [id=81d999d094e04c55a3cf54ab5b5f826b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:29.237754 13430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea0c8d11e39405cb824f1375ab71b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:29.240083 14095 tablet_bootstrap.cc:492] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095: Bootstrap starting.
I20260812 21:08:29.240489 14097 tablet_bootstrap.cc:492] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: Bootstrap starting.
I20260812 21:08:29.240916 14095 tablet_bootstrap.cc:654] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:29.241276 14097 tablet_bootstrap.cc:654] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:29.241477 14098 tablet_bootstrap.cc:492] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba: Bootstrap starting.
I20260812 21:08:29.241892 14095 log.cc:824] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:29.242092 14097 log.cc:824] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:29.242600 14098 tablet_bootstrap.cc:654] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:29.242807 14095 tablet_bootstrap.cc:492] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095: No bootstrap required, opened a new log
I20260812 21:08:29.242901 14097 tablet_bootstrap.cc:492] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: No bootstrap required, opened a new log
I20260812 21:08:29.242926 14095 ts_tablet_manager.cc:1397] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 21:08:29.242960 14097 ts_tablet_manager.cc:1397] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:08:29.243256 14098 log.cc:824] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:29.243891 14098 tablet_bootstrap.cc:492] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba: No bootstrap required, opened a new log
I20260812 21:08:29.243948 14098 ts_tablet_manager.cc:1397] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:08:29.246070 14095 raft_consensus.cc:348] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.246260 14095 raft_consensus.cc:374] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:29.246299 14095 raft_consensus.cc:729] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb31ca1b57e4fd0bdf072b6b2cc9095, State: Initialized, Role: FOLLOWER
I20260812 21:08:29.246145 14098 raft_consensus.cc:348] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.246208 14097 raft_consensus.cc:348] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.246366 14097 raft_consensus.cc:374] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:29.246408 14097 raft_consensus.cc:729] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d97325d928d411d9397a6ae3b986ce9, State: Initialized, Role: FOLLOWER
I20260812 21:08:29.246362 14098 raft_consensus.cc:374] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:29.246443 14098 raft_consensus.cc:729] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d0335e843b468f9bb678115ab98eba, State: Initialized, Role: FOLLOWER
I20260812 21:08:29.246428 14095 consensus_queue.cc:260] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.246519 14097 consensus_queue.cc:260] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.246574 14098 consensus_queue.cc:260] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.246785 14095 ts_tablet_manager.cc:1428] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 21:08:29.246799 14097 ts_tablet_manager.cc:1428] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 21:08:29.246862 14065 heartbeater.cc:503] Master 127.11.207.190:42077 was elected leader, sending a full tablet report...
I20260812 21:08:29.246917 13813 heartbeater.cc:503] Master 127.11.207.190:42077 was elected leader, sending a full tablet report...
I20260812 21:08:29.247309 14098 ts_tablet_manager.cc:1428] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:08:29.247438 13561 heartbeater.cc:503] Master 127.11.207.190:42077 was elected leader, sending a full tablet report...
I20260812 21:08:29.289060 14107 raft_consensus.cc:482] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:29.289191 14107 raft_consensus.cc:504] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.289834 14107 leader_election.cc:288] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d97325d928d411d9397a6ae3b986ce9 (127.11.207.130:35077), 67d0335e843b468f9bb678115ab98eba (127.11.207.129:34793)
I20260812 21:08:29.293064 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.290219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:35279 (local address 127.11.207.129:34793)
0812 21:08:29.290280 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.290282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.290291 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:29.290317 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.290318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.290326 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.290390 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:29.290637 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.291203 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.292165 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.292630 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.292749 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.292762 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.292951 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.292992 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:08:29.293064 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.290011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:34793 (local address 127.11.207.131:35279)
0812 21:08:29.290070 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.290082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.290125 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.290413 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.290414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.290416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:29.290556 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.290559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.291307 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.291308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.292076 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.292079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.292753 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.292754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.292879 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.292883 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.292933 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.292978 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:08:29.293155 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.290178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:44987 (local address 127.11.207.130:35077)
0812 21:08:29.290252 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.290254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.290501 (+ 247us) server_negotiation.cc:407] Connection header received
0812 21:08:29.290558 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.290560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.290570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.290617 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:29.290783 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.291244 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.292369 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.292745 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.292863 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.292874 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.293039 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.293084 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:08:29.293210 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.290011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:35077 (local address 127.11.207.131:44987)
0812 21:08:29.290285 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:29.290407 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:29.290498 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:29.290623 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:29.290624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:29.290625 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:29.290719 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.290721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.291317 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.291318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.292164 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:29.292167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.292870 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.292871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:29.292966 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.292970 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:29.293026 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:08:29.293067 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":23,"thread_start_us":145,"threads_started":1}
I20260812 21:08:29.293612 13724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0c8d11e39405cb824f1375ab71b8c" candidate_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d97325d928d411d9397a6ae3b986ce9" is_pre_election: true
I20260812 21:08:29.293843 13724 raft_consensus.cc:2454] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bb31ca1b57e4fd0bdf072b6b2cc9095 in term 0.
I20260812 21:08:29.293635 13473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0c8d11e39405cb824f1375ab71b8c" candidate_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d0335e843b468f9bb678115ab98eba" is_pre_election: true
I20260812 21:08:29.293969 13473 raft_consensus.cc:2454] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bb31ca1b57e4fd0bdf072b6b2cc9095 in term 0.
I20260812 21:08:29.294198 13846 leader_election.cc:302] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb31ca1b57e4fd0bdf072b6b2cc9095, 5d97325d928d411d9397a6ae3b986ce9; no voters:
I20260812 21:08:29.294320 14107 raft_consensus.cc:2781] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:29.294385 14107 raft_consensus.cc:482] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:29.294402 14107 raft_consensus.cc:3037] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:29.295428 14107 raft_consensus.cc:504] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.295619 14107 leader_election.cc:288] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [CANDIDATE]: Term 1 election: Requested vote from peers 5d97325d928d411d9397a6ae3b986ce9 (127.11.207.130:35077), 67d0335e843b468f9bb678115ab98eba (127.11.207.129:34793)
I20260812 21:08:29.295696 13724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0c8d11e39405cb824f1375ab71b8c" candidate_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d97325d928d411d9397a6ae3b986ce9"
I20260812 21:08:29.295738 13473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea0c8d11e39405cb824f1375ab71b8c" candidate_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d0335e843b468f9bb678115ab98eba"
I20260812 21:08:29.295759 13724 raft_consensus.cc:3037] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:29.295796 13473 raft_consensus.cc:3037] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [term 0 FOLLOWER]: Advancing to term 1
W20260812 21:08:29.296581 13814 tablet.cc:2369] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:29.296615 13473 raft_consensus.cc:2454] T 3ea0c8d11e39405cb824f1375ab71b8c P 67d0335e843b468f9bb678115ab98eba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bb31ca1b57e4fd0bdf072b6b2cc9095 in term 1.
I20260812 21:08:29.296888 13847 leader_election.cc:302] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bb31ca1b57e4fd0bdf072b6b2cc9095, 67d0335e843b468f9bb678115ab98eba; no voters:
I20260812 21:08:29.296926 13724 raft_consensus.cc:2454] T 3ea0c8d11e39405cb824f1375ab71b8c P 5d97325d928d411d9397a6ae3b986ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bb31ca1b57e4fd0bdf072b6b2cc9095 in term 1.
I20260812 21:08:29.297006 14107 raft_consensus.cc:2781] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:29.297201 14107 raft_consensus.cc:686] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [term 1 LEADER]: Becoming Leader. State: Replica: 4bb31ca1b57e4fd0bdf072b6b2cc9095, State: Running, Role: LEADER
I20260812 21:08:29.297317 14107 consensus_queue.cc:237] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } }
I20260812 21:08:29.298239 13202 catalog_manager.cc:5495] T 3ea0c8d11e39405cb824f1375ab71b8c P 4bb31ca1b57e4fd0bdf072b6b2cc9095 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bb31ca1b57e4fd0bdf072b6b2cc9095 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d97325d928d411d9397a6ae3b986ce9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d0335e843b468f9bb678115ab98eba" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bb31ca1b57e4fd0bdf072b6b2cc9095" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 35973 } health_report { overall_health: HEALTHY } } }
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:29.319439 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.312530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.11.207.129:34793)
0812 21:08:29.312594 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.312597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.312682 (+ 85us) server_negotiation.cc:407] Connection header received
0812 21:08:29.312751 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.312752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.312763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.312812 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:29.313337 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.313819 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.314514 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.314713 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.314769 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.316973 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:29.316988 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:29.316988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:29.316996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:29.318403 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.318706 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.318710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.318710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.318764 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.318927 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.318928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.318929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.319074 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:29.319202 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.319263 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.319344 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:29.342257 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.335942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.11.207.129:34793)
0812 21:08:29.336031 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.336033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.336048 (+ 15us) server_negotiation.cc:407] Connection header received
0812 21:08:29.336110 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.336111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.336120 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.336165 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:29.336722 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.337224 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.337873 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.338065 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.338113 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.340092 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:29.340094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:29.340094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:29.340101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:29.341420 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.341682 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.341683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.341683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.341732 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.341849 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.341850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.341850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.341933 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:29.341997 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.342105 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.342162 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[13119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:29.365798 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:29.359430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.11.207.129:34793)
0812 21:08:29.359494 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:29.359496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:29.359553 (+ 57us) server_negotiation.cc:407] Connection header received
0812 21:08:29.359607 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:29.359608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:29.359617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:29.359665 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:29.360196 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.360709 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.361418 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:29.361596 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:29.361635 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:29.363709 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:29.363711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:29.363711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:29.363718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:29.364865 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.365223 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.365224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.365225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.365274 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:29.365429 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:29.365429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:29.365430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:29.365512 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:29.365578 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:29.365673 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:08:29.365730 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 21:08:29.366694 13399 tablet_service.cc:2908] Rejecting scan request for tablet 3ea0c8d11e39405cb824f1375ab71b8c: Uninitialized: clean time has not yet been initialized
I20260812 21:08:29.367663 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13314
I20260812 21:08:29.385962 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13570
I20260812 21:08:29.400063 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13820
I20260812 21:08:29.411237 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13146
2026-08-12T21:08:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2175 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[14136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[14136](info): set up 2 sockets
Aug 12 21:08:29 dist-test-slave-lkx2 krb5kdc[14136](info): commencing operation
krb5kdc: starting...
W20260812 21:08:30.617054 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:30 dist-test-slave-lkx2 krb5kdc[14136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:30.737326 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43825
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:30.825352 14163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:30.825650 14163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:30.825735 14163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:30.827275 14163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:30.827322 14163 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 21:08:30.827335 14163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:30.827347 14163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:30.827358 14163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:30.827369 14163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:30.828923 14163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43825
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:30.829133 14163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:30.829322 14163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:30.834153 14172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:30.834219 14163 server_base.cc:1034] running on GCE node
W20260812 21:08:30.834153 14174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:30.834177 14176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:30.834708 14163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:30.835011 14163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:30.836141 14163 hybrid_clock.cc:648] HybridClock initialized: now 1786568910836116 us; error 45 us; skew 500 ppm
Aug 12 21:08:30 dist-test-slave-lkx2 krb5kdc[14136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568910, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:30.843717 14163 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:30.844165 14163 webserver.cc:460] Webserver started at http://127.11.207.190:33943/ using document root <none> and password file <none>
I20260812 21:08:30.844396 14163 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:30.844444 14163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:30.844545 14163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:30.845624 14163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "a9b9e0cd23164ae8a97802eecc039d91"
format_stamp: "Formatted at 2026-08-12 21:08:30 on dist-test-slave-lkx2"
server_key: "6e530e83c8be53021e5458c61c07001d"
server_key_iv: "df6d250c125b916b7ecb70e328dac7fa"
server_key_version: "encryptionkey@0"
I20260812 21:08:30.845937 14163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "a9b9e0cd23164ae8a97802eecc039d91"
format_stamp: "Formatted at 2026-08-12 21:08:30 on dist-test-slave-lkx2"
server_key: "6e530e83c8be53021e5458c61c07001d"
server_key_iv: "df6d250c125b916b7ecb70e328dac7fa"
server_key_version: "encryptionkey@0"
I20260812 21:08:30.847241 14163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:30.848021 14194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:30.848192 14163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:30.848281 14163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "a9b9e0cd23164ae8a97802eecc039d91"
format_stamp: "Formatted at 2026-08-12 21:08:30 on dist-test-slave-lkx2"
server_key: "6e530e83c8be53021e5458c61c07001d"
server_key_iv: "df6d250c125b916b7ecb70e328dac7fa"
server_key_version: "encryptionkey@0"
I20260812 21:08:30.848372 14163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:30.864619 14163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:30.866497 14163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:30.866611 14163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:30.871297 14163 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43825
I20260812 21:08:30.871377 14301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43825 every 8 connection(s)
I20260812 21:08:30.871685 14163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:30.872406 14302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:30.872818 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14163
I20260812 21:08:30.872915 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:30.873078 12094 external_mini_cluster.cc:1428] Setting key 447924a9e2947928347e72ec362d2a37
I20260812 21:08:30.875489 14302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: Bootstrap starting.
I20260812 21:08:30.876134 14302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:30.876492 14302 log.cc:824] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:30.877250 14302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: No bootstrap required, opened a new log
Aug 12 21:08:30 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:30.879522 14302 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b9e0cd23164ae8a97802eecc039d91" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43825 } }
I20260812 21:08:30.879668 14302 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:30.879705 14302 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b9e0cd23164ae8a97802eecc039d91, State: Initialized, Role: FOLLOWER
I20260812 21:08:30.879824 14302 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b9e0cd23164ae8a97802eecc039d91" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43825 } }
I20260812 21:08:30.879889 14302 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:30.879917 14302 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:30.879966 14302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:30.880628 14302 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b9e0cd23164ae8a97802eecc039d91" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43825 } }
I20260812 21:08:30.880752 14302 leader_election.cc:302] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9b9e0cd23164ae8a97802eecc039d91; no voters:
I20260812 21:08:30.880939 14302 leader_election.cc:288] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:30.880976 14309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:30.881145 14309 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [term 1 LEADER]: Becoming Leader. State: Replica: a9b9e0cd23164ae8a97802eecc039d91, State: Running, Role: LEADER
I20260812 21:08:30.881217 14302 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:30.882565 14309 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b9e0cd23164ae8a97802eecc039d91" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43825 } }
I20260812 21:08:30.883170 14310 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b9e0cd23164ae8a97802eecc039d91. Latest consensus state: current_term: 1 leader_uuid: "a9b9e0cd23164ae8a97802eecc039d91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b9e0cd23164ae8a97802eecc039d91" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43825 } } }
I20260812 21:08:30.883263 14310 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:30.883462 14332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:30.883491 14313 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9b9e0cd23164ae8a97802eecc039d91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9b9e0cd23164ae8a97802eecc039d91" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43825 } } }
I20260812 21:08:30.883580 14313 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:30.883651 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:30.873627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.11.207.190:43825)
0812 21:08:30.873776 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:30.873784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:30.873810 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:08:30.874222 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:30.874229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:30.874275 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:30.874397 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:30.875102 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:30.876022 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:30.877011 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:30.877440 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:30.877520 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:30.879731 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:30.879735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:30.879736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:30.879749 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:30.881681 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:30.882114 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:30.882116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:30.882117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:30.882211 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:30.882399 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:30.882400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:30.882400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:30.882641 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:30.882778 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:30.883215 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 21:08:30.883370 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260812 21:08:30.884111 14332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:30.885566 14332 catalog_manager.cc:1347] Generated new cluster ID: dc4ccb5f6ab04d73858053f847bc8f73
I20260812 21:08:30.885608 14332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:30.892694 14332 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:30.893211 14332 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:30.900079 14332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: Generated new TSK 0
I20260812 21:08:30.900256 14332 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:30.939277 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43825
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:31.034910 14340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:31.035140 14340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:31.035176 14340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:31.036460 14340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:31.036505 14340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:31.036535 14340 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:08:31.036561 14340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:31.038105 14340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:31.038362 14340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:31.038587 14340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:31.043664 14347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:31.043803 14346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:31.043840 14340 server_base.cc:1034] running on GCE node
W20260812 21:08:31.043773 14349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:31.044162 14340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:31.044442 14340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:31.045598 14340 hybrid_clock.cc:648] HybridClock initialized: now 1786568911045584 us; error 29 us; skew 500 ppm
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:31.048249 14340 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:31.048702 14340 webserver.cc:460] Webserver started at http://127.11.207.129:45297/ using document root <none> and password file <none>
I20260812 21:08:31.048936 14340 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:31.048980 14340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:31.049077 14340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:31.050164 14340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "996acf083aa447adb2bab1f81c06ebf4"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "13cf50ee8ec753dcc70087e8ada65daa"
server_key_iv: "083f672222bd6e7b5befcffe3c732e9e"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.050457 14340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "996acf083aa447adb2bab1f81c06ebf4"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "13cf50ee8ec753dcc70087e8ada65daa"
server_key_iv: "083f672222bd6e7b5befcffe3c732e9e"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.051699 14340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:31.052431 14361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.052587 14340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.052671 14340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "996acf083aa447adb2bab1f81c06ebf4"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "13cf50ee8ec753dcc70087e8ada65daa"
server_key_iv: "083f672222bd6e7b5befcffe3c732e9e"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.052731 14340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:31.075187 14340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:31.077091 14340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:31.077221 14340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:31.077764 14340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:31.077802 14340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.077823 14340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:31.077836 14340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.084978 14340 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:41977
I20260812 21:08:31.085042 14588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:41977 every 8 connection(s)
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:31.085412 14340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:31.086032 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14340
I20260812 21:08:31.086133 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:31.086290 12094 external_mini_cluster.cc:1428] Setting key 39e57ac4a4ed79f6ed2aadc2878c7780
I20260812 21:08:31.088862 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.078475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:38699 (local address 127.11.207.190:43825)
0812 21:08:31.078539 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.078541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.078738 (+ 197us) server_negotiation.cc:407] Connection header received
0812 21:08:31.079318 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.079319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.079332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.079408 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.080358 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.081132 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.082835 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.083124 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.083182 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.085975 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.085976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.085977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.085985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.087491 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.087877 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.087878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.087879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.087938 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.088165 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.088166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.088166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.088281 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.088376 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.088699 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.088796 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:08:31.089069 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.078426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.11.207.129:38699)
0812 21:08:31.078598 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.078642 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.079204 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.079523 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.079528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.079930 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.080234 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.080243 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.081253 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.081254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.082693 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.082697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.083212 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.083213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.083368 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.084004 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.084015 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.085860 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.087570 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.087579 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.087581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.087809 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.087994 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.087995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.087995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.088107 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.088337 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.088365 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.088539 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.088661 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.088844 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260812 21:08:31.089605 14594 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43825
I20260812 21:08:31.089704 14594 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:31.089944 14594 heartbeater.cc:511] Master 127.11.207.190:43825 requested a full tablet report, sending...
I20260812 21:08:31.090541 14218 ts_manager.cc:194] Registered new tserver with Master: 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977)
I20260812 21:08:31.091401 14218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:38699
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
I20260812 21:08:31.096925 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.090840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.11.207.129:45285)
0812 21:08:31.090987 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.090996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.091050 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.091163 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.091164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.091354 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.091429 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.091432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.092389 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.092390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.092908 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.092910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.093350 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.093351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.093429 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.093872 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.093877 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.094205 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.095922 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.095923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.095924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.096089 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.096280 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.096280 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.096281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.096332 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.096612 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.096613 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.096664 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.096710 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.096806 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260812 21:08:31.096930 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.090890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:45285 (local address 127.11.207.190:43825)
0812 21:08:31.090925 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.090926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.091048 (+ 122us) server_negotiation.cc:407] Connection header received
0812 21:08:31.091095 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.091096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.091105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.091161 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.091564 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.092269 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.092986 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.093280 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.093349 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.094289 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.094299 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.094300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.094307 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.095838 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.096159 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.096161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.096161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.096219 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.096412 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.096412 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.096413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.096552 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.096613 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.096755 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.096839 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:31.114557 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43825
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:31.198709 14605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:31.198961 14605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:31.198998 14605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:31.200233 14605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:31.200284 14605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:31.200313 14605 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:08:31.200336 14605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:31.201910 14605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:31.202157 14605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:31.202382 14605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:31.207532 14612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:31.207573 14605 server_base.cc:1034] running on GCE node
W20260812 21:08:31.207541 14617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:31.207571 14620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:31.207978 14605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:31.208199 14605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:31.209347 14605 hybrid_clock.cc:648] HybridClock initialized: now 1786568911209331 us; error 33 us; skew 500 ppm
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:31.211880 14605 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:31.212311 14605 webserver.cc:460] Webserver started at http://127.11.207.130:42711/ using document root <none> and password file <none>
I20260812 21:08:31.212538 14605 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:31.212587 14605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:31.212715 14605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:31.213791 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "33f1abd6392a4b8480a838641f5466d1"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "ceca3d93606c1a0221ab818f593648c4"
server_key_iv: "31771355e77b89d0e71f9cd3698ec718"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.214133 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "33f1abd6392a4b8480a838641f5466d1"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "ceca3d93606c1a0221ab818f593648c4"
server_key_iv: "31771355e77b89d0e71f9cd3698ec718"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.215394 14605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:31.216049 14629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.216193 14605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:31.216292 14605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "33f1abd6392a4b8480a838641f5466d1"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "ceca3d93606c1a0221ab818f593648c4"
server_key_iv: "31771355e77b89d0e71f9cd3698ec718"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.216370 14605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:31.230680 14605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:31.232609 14605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:31.232748 14605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:31.233275 14605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:31.233315 14605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.233353 14605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:31.233378 14605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:31.240327 14605 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:44445
I20260812 21:08:31.240463 14842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:44445 every 8 connection(s)
I20260812 21:08:31.240705 14605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:31.243315 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.233834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:37519 (local address 127.11.207.190:43825)
0812 21:08:31.233906 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.233908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.234124 (+ 216us) server_negotiation.cc:407] Connection header received
0812 21:08:31.234716 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.234718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.234731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.234800 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.235672 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.236451 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.237949 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.238262 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.238336 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.240875 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.240876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.240877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.240886 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.242204 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.242577 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.242578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.242578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.242625 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.242831 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.242831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.242831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.242928 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.242972 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.243194 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.243252 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:08:31.243506 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.233802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.11.207.130:37519)
0812 21:08:31.233991 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.234035 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.234580 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.234908 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.234914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.235272 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.235579 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.235586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.236551 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.236552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.237864 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.237868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.238361 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.238362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.238534 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.239154 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.239164 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.240773 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.242278 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.242285 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.242287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.242515 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.242679 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.242680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.242680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.242780 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.242972 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.242976 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.243086 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.243181 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.243305 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260812 21:08:31.243929 14843 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43825
I20260812 21:08:31.244009 14843 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:31.244177 14843 heartbeater.cc:511] Master 127.11.207.190:43825 requested a full tablet report, sending...
I20260812 21:08:31.244571 14216 ts_manager.cc:194] Registered new tserver with Master: 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130:44445)
I20260812 21:08:31.245101 14216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:37519
I20260812 21:08:31.249900 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.245044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:40405 (local address 127.11.207.190:43825)
0812 21:08:31.245080 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.245082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.245154 (+ 72us) server_negotiation.cc:407] Connection header received
0812 21:08:31.245265 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.245266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.245276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.245325 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.245675 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.246199 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.246859 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.247027 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.247069 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.247833 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.247833 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.247834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.247839 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.249003 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.249331 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.249332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.249332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.249388 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.249518 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.249519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.249519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.249641 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.249688 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.249787 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.249843 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:08:31.249917 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.244961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.11.207.130:40405)
0812 21:08:31.245084 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.245096 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.245172 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.245339 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.245340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.245541 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.245604 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.245605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.246285 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.246285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.246790 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.246792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.247077 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.247078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.247138 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.247479 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.247483 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.247755 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.249091 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.249092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.249092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.249273 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.249432 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.249432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.249433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.249480 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.249683 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.249683 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.249717 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.249771 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.249848 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":60,"threads_started":1}
I20260812 21:08:31.250155 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14605
I20260812 21:08:31.250237 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:31.250449 12094 external_mini_cluster.cc:1428] Setting key e4e017b94a4630280b81aba5731c62ee
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:31.275557 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43825
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:31.358263 14857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:31.358484 14857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:31.358507 14857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:31.359695 14857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:31.359742 14857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:31.359763 14857 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:08:31.359795 14857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:31.361286 14857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:31.361523 14857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:31.361743 14857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:31.366744 14870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:31.366753 14874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:31.366748 14871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:31.366940 14857 server_base.cc:1034] running on GCE node
I20260812 21:08:31.367234 14857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:31.367472 14857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:31.368608 14857 hybrid_clock.cc:648] HybridClock initialized: now 1786568911368594 us; error 31 us; skew 500 ppm
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:31.371277 14857 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:31.371733 14857 webserver.cc:460] Webserver started at http://127.11.207.131:41255/ using document root <none> and password file <none>
I20260812 21:08:31.371943 14857 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:31.371994 14857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:31.372121 14857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:31.373260 14857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "b9ce6a6d7b1744988ba60ee983de8724"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "c2efc26c30fd8d30bc7202d4ff588173"
server_key_iv: "dbd5ac56674e1f5b564e2763eb4dfa46"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.373626 14857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "b9ce6a6d7b1744988ba60ee983de8724"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "c2efc26c30fd8d30bc7202d4ff588173"
server_key_iv: "dbd5ac56674e1f5b564e2763eb4dfa46"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.375033 14857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:31.375749 14884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.375900 14857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:31.375989 14857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "b9ce6a6d7b1744988ba60ee983de8724"
format_stamp: "Formatted at 2026-08-12 21:08:31 on dist-test-slave-lkx2"
server_key: "c2efc26c30fd8d30bc7202d4ff588173"
server_key_iv: "dbd5ac56674e1f5b564e2763eb4dfa46"
server_key_version: "encryptionkey@0"
I20260812 21:08:31.376056 14857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:31.396921 14857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:31.398998 14857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:31.399147 14857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:31.399705 14857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:31.399747 14857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.399782 14857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:31.399811 14857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:31.407648 14857 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:46375
I20260812 21:08:31.407778 15091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:46375 every 8 connection(s)
I20260812 21:08:31.408074 14857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:31.410069 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.400277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:55889 (local address 127.11.207.190:43825)
0812 21:08:31.400331 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.400332 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.400639 (+ 307us) server_negotiation.cc:407] Connection header received
0812 21:08:31.401349 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.401350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.401362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.401409 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.402329 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.403010 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.404528 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.404809 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.404880 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.407145 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.407147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.407147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.407156 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.408786 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.409163 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.409164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.409164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.409221 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.409535 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.409535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.409535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.409639 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.409686 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.409938 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.410005 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:31.410238 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.400277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.11.207.131:55889)
0812 21:08:31.400502 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.400554 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.401230 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.401497 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.401501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.401887 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.402184 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.402192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.403130 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.403131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.404443 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.404446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.404902 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.404903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.405033 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.405525 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.405536 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.407043 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.408881 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.408885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.408887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.409103 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.409261 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.409262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.409262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.409467 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.409687 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.409692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.409831 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.409919 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.410048 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":126,"threads_started":1}
I20260812 21:08:31.410907 15092 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43825
I20260812 21:08:31.410997 15092 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:31.411056 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14857
I20260812 21:08:31.411149 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:31.411201 15092 heartbeater.cc:511] Master 127.11.207.190:43825 requested a full tablet report, sending...
I20260812 21:08:31.411315 12094 external_mini_cluster.cc:1428] Setting key e8c5e8461ad7a71a965828fed572ab59
I20260812 21:08:31.411898 14218 ts_manager.cc:194] Registered new tserver with Master: b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375)
I20260812 21:08:31.412364 14218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:55889
I20260812 21:08:31.417377 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.412566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.11.207.131:50677)
0812 21:08:31.412699 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.412709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.412761 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.412878 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.412878 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.413034 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.413097 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.413098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.413708 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.413709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.414225 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.414226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.414536 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.414537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.414587 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.414990 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.414995 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.415288 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.416579 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.416580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.416580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.416734 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.416939 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.416939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.416939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.416985 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.417149 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.417150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.417192 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.417233 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.417308 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260812 21:08:31.417377 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.412602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:50677 (local address 127.11.207.190:43825)
0812 21:08:31.412645 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.412646 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.412762 (+ 116us) server_negotiation.cc:407] Connection header received
0812 21:08:31.412797 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.412798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.412809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.412873 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.413159 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.413629 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.414296 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.414480 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.414533 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.415375 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.415376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.415376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.415382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.416517 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.416814 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.416815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.416816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.416875 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.417021 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.417021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.417021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.417116 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.417161 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.417259 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.417319 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:31.422947 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:31.446063 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.439438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.11.207.190:43825)
0812 21:08:31.439495 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.439497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.439504 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:31.439530 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.439530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.439540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.439588 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.440137 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.440726 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.441490 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.441749 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.441812 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.443955 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.443957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.443957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.443965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.445138 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.445433 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.445433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.445434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.445483 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.445598 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.445598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.445598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.445723 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.445829 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.445931 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.445995 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:08:31.447657 14218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:31.449368 14218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:31.455891 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.451810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:46375 (local address 127.0.0.1:52814)
0812 21:08:31.452307 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.452315 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.452384 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.452610 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.452611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.452614 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.452691 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.452693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.453942 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.453944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.454965 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.454967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.455619 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.455696 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.455710 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.455773 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.455819 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":27,"thread_start_us":74,"threads_started":1}
I20260812 21:08:31.456164 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.452085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.11.207.129:41977)
0812 21:08:31.452330 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.452334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.452367 (+ 33us) server_negotiation.cc:407] Connection header received
0812 21:08:31.452408 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.452410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.452428 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.452499 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.452744 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.453857 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455096 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.455851 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455922 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.455969 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.456053 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.456090 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":75,"threads_started":1}
I20260812 21:08:31.456322 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.452064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.11.207.131:46375)
0812 21:08:31.452251 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.452256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.452420 (+ 164us) server_negotiation.cc:407] Connection header received
0812 21:08:31.452463 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.452465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.452480 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.452548 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.452771 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.453857 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455035 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.455547 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455613 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.455672 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.456194 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.456235 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260812 21:08:31.456378 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.451907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50924 (local address 127.11.207.130:44445)
0812 21:08:31.452283 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.452287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.452298 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:31.452493 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.452496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.452513 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.452580 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.452759 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.453474 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.454782 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.455573 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455641 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.455705 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.456209 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.456252 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":90,"threads_started":1}
I20260812 21:08:31.456029 15125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.451787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44445 (local address 127.0.0.1:50924)
0812 21:08:31.452077 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.452120 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.452206 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.452645 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.452646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.452648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.452688 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.452689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.453548 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.453549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.454713 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.454717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.455790 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.455853 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.455858 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.455908 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.455953 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":63,"threads_started":1}
I20260812 21:08:31.457425 14978 tablet_service.cc:1467] Processing CreateTablet for tablet e53a04890ddd43c3a1aca53e7d280585 (DEFAULT_TABLE table=test-table [id=60f1e7eea43b4d82bef16c73e2583e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:31.457384 14720 tablet_service.cc:1467] Processing CreateTablet for tablet e53a04890ddd43c3a1aca53e7d280585 (DEFAULT_TABLE table=test-table [id=60f1e7eea43b4d82bef16c73e2583e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:31.457804 14720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53a04890ddd43c3a1aca53e7d280585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.457805 14978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53a04890ddd43c3a1aca53e7d280585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.456141 15123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.451793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:41977 (local address 127.0.0.1:36730)
0812 21:08:31.452240 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.452249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.452290 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.452599 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.452602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.452605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.452674 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.452676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.453955 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.453956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.455002 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.455005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.455924 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.455925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.455981 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.455986 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.456040 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.456073 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":36,"thread_start_us":99,"threads_started":1}
I20260812 21:08:31.461100 14464 tablet_service.cc:1467] Processing CreateTablet for tablet e53a04890ddd43c3a1aca53e7d280585 (DEFAULT_TABLE table=test-table [id=60f1e7eea43b4d82bef16c73e2583e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:31.461349 15136 tablet_bootstrap.cc:492] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Bootstrap starting.
I20260812 21:08:31.461503 14464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53a04890ddd43c3a1aca53e7d280585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.461774 15137 tablet_bootstrap.cc:492] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Bootstrap starting.
I20260812 21:08:31.462322 15137 tablet_bootstrap.cc:654] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.462375 15136 tablet_bootstrap.cc:654] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.462561 15137 log.cc:824] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:31.462955 15136 log.cc:824] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:31.463358 15137 tablet_bootstrap.cc:492] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: No bootstrap required, opened a new log
I20260812 21:08:31.463407 15137 ts_tablet_manager.cc:1397] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:31.465090 15137 raft_consensus.cc:348] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.465220 15137 raft_consensus.cc:374] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.465220 15144 tablet_bootstrap.cc:492] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Bootstrap starting.
I20260812 21:08:31.465253 15137 raft_consensus.cc:729] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f1abd6392a4b8480a838641f5466d1, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.465370 15137 consensus_queue.cc:260] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.465695 15137 ts_tablet_manager.cc:1428] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:08:31.465751 14843 heartbeater.cc:503] Master 127.11.207.190:43825 was elected leader, sending a full tablet report...
I20260812 21:08:31.466161 15144 tablet_bootstrap.cc:654] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.466339 15136 tablet_bootstrap.cc:492] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: No bootstrap required, opened a new log
I20260812 21:08:31.466408 15136 ts_tablet_manager.cc:1397] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 21:08:31.466666 15144 log.cc:824] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:31.467556 15144 tablet_bootstrap.cc:492] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: No bootstrap required, opened a new log
I20260812 21:08:31.467626 15144 ts_tablet_manager.cc:1397] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:31.468860 15136 raft_consensus.cc:348] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.469012 15136 raft_consensus.cc:374] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.469046 15136 raft_consensus.cc:729] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ce6a6d7b1744988ba60ee983de8724, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.469164 15136 consensus_queue.cc:260] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.469476 15136 ts_tablet_manager.cc:1428] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:08:31.469545 15092 heartbeater.cc:503] Master 127.11.207.190:43825 was elected leader, sending a full tablet report...
I20260812 21:08:31.469566 15144 raft_consensus.cc:348] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.469719 15144 raft_consensus.cc:374] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.469762 15144 raft_consensus.cc:729] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996acf083aa447adb2bab1f81c06ebf4, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.469869 15144 consensus_queue.cc:260] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.470130 15144 ts_tablet_manager.cc:1428] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:08:31.470163 14594 heartbeater.cc:503] Master 127.11.207.190:43825 was elected leader, sending a full tablet report...
W20260812 21:08:31.491328 14846 tablet.cc:2369] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:31.544747 15147 raft_consensus.cc:482] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:31.544884 15147 raft_consensus.cc:504] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.545374 15147 leader_election.cc:288] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977), 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130:44445)
I20260812 21:08:31.549270 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.545664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:56503 (local address 127.11.207.130:44445)
0812 21:08:31.545719 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.545721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.545730 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:31.545757 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.545759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.545767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.545816 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.546094 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.546851 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.548307 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.548913 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.548986 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.548995 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.549153 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.549195 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:08:31.549271 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.545635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:42605 (local address 127.11.207.129:41977)
0812 21:08:31.545678 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.545680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.545841 (+ 161us) server_negotiation.cc:407] Connection header received
0812 21:08:31.545901 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.545902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.545909 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.545948 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.546094 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.546845 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.548306 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.548910 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.548986 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.548995 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.549153 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.549211 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:31.549304 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.545557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44445 (local address 127.11.207.131:56503)
0812 21:08:31.545615 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.545627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.545671 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.545839 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.545840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.545842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.546017 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.546020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.546927 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.546928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.548203 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.548206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.548987 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.548987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.549094 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.549096 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.549143 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.549195 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":92}
I20260812 21:08:31.549309 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.545569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:41977 (local address 127.11.207.131:42605)
0812 21:08:31.545768 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.545777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.545831 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.545969 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.545970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.545971 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.546038 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.546039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.546927 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.546928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.548203 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.548206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.548987 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.548988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.549082 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.549086 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.549143 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.549195 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":13,"thread_start_us":116,"threads_started":1}
I20260812 21:08:31.549752 14505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53a04890ddd43c3a1aca53e7d280585" candidate_uuid: "b9ce6a6d7b1744988ba60ee983de8724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996acf083aa447adb2bab1f81c06ebf4" is_pre_election: true
I20260812 21:08:31.549795 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53a04890ddd43c3a1aca53e7d280585" candidate_uuid: "b9ce6a6d7b1744988ba60ee983de8724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f1abd6392a4b8480a838641f5466d1" is_pre_election: true
I20260812 21:08:31.549947 14758 raft_consensus.cc:2454] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ce6a6d7b1744988ba60ee983de8724 in term 0.
I20260812 21:08:31.549913 14505 raft_consensus.cc:2454] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ce6a6d7b1744988ba60ee983de8724 in term 0.
I20260812 21:08:31.550204 14891 leader_election.cc:302] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f1abd6392a4b8480a838641f5466d1, b9ce6a6d7b1744988ba60ee983de8724; no voters:
I20260812 21:08:31.550400 15147 raft_consensus.cc:2781] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:31.550472 15147 raft_consensus.cc:482] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:31.550496 15147 raft_consensus.cc:3037] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:31.551366 15147 raft_consensus.cc:504] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.551498 15147 leader_election.cc:288] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [CANDIDATE]: Term 1 election: Requested vote from peers 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977), 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130:44445)
I20260812 21:08:31.551690 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53a04890ddd43c3a1aca53e7d280585" candidate_uuid: "b9ce6a6d7b1744988ba60ee983de8724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f1abd6392a4b8480a838641f5466d1"
I20260812 21:08:31.551690 14505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53a04890ddd43c3a1aca53e7d280585" candidate_uuid: "b9ce6a6d7b1744988ba60ee983de8724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996acf083aa447adb2bab1f81c06ebf4"
I20260812 21:08:31.551780 14505 raft_consensus.cc:3037] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:31.551780 14758 raft_consensus.cc:3037] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:31.552888 14505 raft_consensus.cc:2454] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ce6a6d7b1744988ba60ee983de8724 in term 1.
I20260812 21:08:31.552974 14758 raft_consensus.cc:2454] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ce6a6d7b1744988ba60ee983de8724 in term 1.
I20260812 21:08:31.553593 14889 leader_election.cc:302] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 996acf083aa447adb2bab1f81c06ebf4, b9ce6a6d7b1744988ba60ee983de8724; no voters:
I20260812 21:08:31.553730 15147 raft_consensus.cc:2781] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:31.553884 15147 raft_consensus.cc:686] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 LEADER]: Becoming Leader. State: Replica: b9ce6a6d7b1744988ba60ee983de8724, State: Running, Role: LEADER
I20260812 21:08:31.553973 15147 consensus_queue.cc:237] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.554824 14218 catalog_manager.cc:5495] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "b9ce6a6d7b1744988ba60ee983de8724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } health_report { overall_health: HEALTHY } } }
I20260812 21:08:31.572883 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.570568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.11.207.131:46375)
0812 21:08:31.570620 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.570622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.570630 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:31.570699 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.570700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.570715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.570771 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:31.570890 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.571361 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.572113 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.572292 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.572337 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.572550 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:31.572658 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:31.572718 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.572756 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.572777 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:08:31.574832 14758 raft_consensus.cc:1261] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Refusing update from remote peer b9ce6a6d7b1744988ba60ee983de8724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:31.574836 14505 raft_consensus.cc:1261] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Refusing update from remote peer b9ce6a6d7b1744988ba60ee983de8724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:31.575201 15157 consensus_queue.cc:1035] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:31.577522 15160 mvcc.cc:205] Tried to move back new op lower bound from 7317786261807513600 to 7317786261725777920. Current Snapshot: MvccSnapshot[applied={T|T < 7317786261807513600}]
I20260812 21:08:31.577638 15147 consensus_queue.cc:1035] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:31.578331 15161 mvcc.cc:205] Tried to move back new op lower bound from 7317786261807513600 to 7317786261725777920. Current Snapshot: MvccSnapshot[applied={T|T < 7317786261807513600 or (T in {7317786261807513600})}]
I20260812 21:08:31.582798 14218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:31.582937 14218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:31.583796 14218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: Sending DeleteTablet for 3 replicas of tablet e53a04890ddd43c3a1aca53e7d280585
I20260812 21:08:31.584232 14464 tablet_service.cc:1514] Processing DeleteTablet for tablet e53a04890ddd43c3a1aca53e7d280585 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:31 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:36730
I20260812 21:08:31.584607 14978 tablet_service.cc:1514] Processing DeleteTablet for tablet e53a04890ddd43c3a1aca53e7d280585 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:31 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:52814
I20260812 21:08:31.584849 15181 tablet_replica.cc:330] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: stopping tablet replica
I20260812 21:08:31.584954 15181 raft_consensus.cc:2229] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:31.584842 14218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:31.585022 14218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:31.585176 15181 raft_consensus.cc:2258] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:31.585940 14720 tablet_service.cc:1514] Processing DeleteTablet for tablet e53a04890ddd43c3a1aca53e7d280585 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:31 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:50924
I20260812 21:08:31.586302 15182 tablet_replica.cc:330] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: stopping tablet replica
I20260812 21:08:31.586514 15180 tablet_replica.cc:330] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: stopping tablet replica
I20260812 21:08:31.586570 14464 tablet_service.cc:1467] Processing CreateTablet for tablet 10af424982bc44bea3c9ae4c3ed61632 (DEFAULT_TABLE table=test-table [id=1e7f9dfd78c046868369b251c7f08204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:31.586676 15180 raft_consensus.cc:2229] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:31.586706 15181 ts_tablet_manager.cc:1905] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:31.586722 14464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10af424982bc44bea3c9ae4c3ed61632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.586781 15180 raft_consensus.cc:2258] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:31.587267 15180 ts_tablet_manager.cc:1905] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:31.588158 15144 tablet_bootstrap.cc:492] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Bootstrap starting.
I20260812 21:08:31.588497 14720 tablet_service.cc:1467] Processing CreateTablet for tablet 10af424982bc44bea3c9ae4c3ed61632 (DEFAULT_TABLE table=test-table [id=1e7f9dfd78c046868369b251c7f08204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:31.588771 14720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10af424982bc44bea3c9ae4c3ed61632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.588881 15144 tablet_bootstrap.cc:654] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.588912 14978 tablet_service.cc:1467] Processing CreateTablet for tablet 10af424982bc44bea3c9ae4c3ed61632 (DEFAULT_TABLE table=test-table [id=1e7f9dfd78c046868369b251c7f08204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:31.590548 15180 ts_tablet_manager.cc:1918] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:31.590615 15180 log.cc:1196] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/e53a04890ddd43c3a1aca53e7d280585
I20260812 21:08:31.590734 15137 tablet_bootstrap.cc:492] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Bootstrap starting.
I20260812 21:08:31.590866 15180 ts_tablet_manager.cc:1939] T e53a04890ddd43c3a1aca53e7d280585 P 996acf083aa447adb2bab1f81c06ebf4: Deleting consensus metadata
I20260812 21:08:31.591229 15137 tablet_bootstrap.cc:654] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.591331 15144 tablet_bootstrap.cc:492] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: No bootstrap required, opened a new log
I20260812 21:08:31.591370 15144 ts_tablet_manager.cc:1397] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 21:08:31.591384 14196 catalog_manager.cc:4841] TS 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977): tablet e53a04890ddd43c3a1aca53e7d280585 (table test-table [id=60f1e7eea43b4d82bef16c73e2583e63]) successfully deleted
I20260812 21:08:31.591500 15144 raft_consensus.cc:348] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.591576 15144 raft_consensus.cc:374] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.591595 15144 raft_consensus.cc:729] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996acf083aa447adb2bab1f81c06ebf4, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.591637 15144 consensus_queue.cc:260] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.591723 15144 ts_tablet_manager.cc:1428] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.592054 15137 tablet_bootstrap.cc:492] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: No bootstrap required, opened a new log
I20260812 21:08:31.592118 15137 ts_tablet_manager.cc:1397] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:31.592253 14978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10af424982bc44bea3c9ae4c3ed61632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.592283 15137 raft_consensus.cc:348] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.592406 15137 raft_consensus.cc:374] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.592449 15137 raft_consensus.cc:729] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f1abd6392a4b8480a838641f5466d1, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.592497 15137 consensus_queue.cc:260] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.592579 15137 ts_tablet_manager.cc:1428] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:31.593745 15136 tablet_bootstrap.cc:492] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Bootstrap starting.
I20260812 21:08:31.593781 15181 ts_tablet_manager.cc:1918] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:31.593852 15181 log.cc:1196] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/e53a04890ddd43c3a1aca53e7d280585
I20260812 21:08:31.594058 15181 ts_tablet_manager.cc:1939] T e53a04890ddd43c3a1aca53e7d280585 P b9ce6a6d7b1744988ba60ee983de8724: Deleting consensus metadata
I20260812 21:08:31.594292 15136 tablet_bootstrap.cc:654] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.594400 15182 raft_consensus.cc:2229] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:31.594527 15182 raft_consensus.cc:2258] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:31.594671 14195 catalog_manager.cc:4841] TS b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375): tablet e53a04890ddd43c3a1aca53e7d280585 (table test-table [id=60f1e7eea43b4d82bef16c73e2583e63]) successfully deleted
I20260812 21:08:31.595047 15182 ts_tablet_manager.cc:1905] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:31.595069 15136 tablet_bootstrap.cc:492] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: No bootstrap required, opened a new log
I20260812 21:08:31.595110 15136 ts_tablet_manager.cc:1397] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:31.595316 15136 raft_consensus.cc:348] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.595393 15136 raft_consensus.cc:374] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.595413 15136 raft_consensus.cc:729] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ce6a6d7b1744988ba60ee983de8724, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.595458 15136 consensus_queue.cc:260] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.595549 15136 ts_tablet_manager.cc:1428] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.596598 15182 ts_tablet_manager.cc:1918] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:31.596658 15182 log.cc:1196] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/e53a04890ddd43c3a1aca53e7d280585
I20260812 21:08:31.596856 15182 ts_tablet_manager.cc:1939] T e53a04890ddd43c3a1aca53e7d280585 P 33f1abd6392a4b8480a838641f5466d1: Deleting consensus metadata
I20260812 21:08:31.597293 14198 catalog_manager.cc:4841] TS 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130:44445): tablet e53a04890ddd43c3a1aca53e7d280585 (table test-table [id=60f1e7eea43b4d82bef16c73e2583e63]) successfully deleted
W20260812 21:08:31.658648 15093 tablet.cc:2369] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:31.808200 15169 raft_consensus.cc:482] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:31.808302 15169 raft_consensus.cc:504] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.808696 15169 leader_election.cc:288] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977), b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375)
I20260812 21:08:31.812786 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.808987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:33981 (local address 127.11.207.131:46375)
0812 21:08:31.809065 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.809067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.809191 (+ 124us) server_negotiation.cc:407] Connection header received
0812 21:08:31.809240 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.809241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.809251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.809309 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.809481 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.810222 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.811800 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.812412 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.812477 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.812485 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.812670 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.812709 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:08:31.812786 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.809002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:43223 (local address 127.11.207.129:41977)
0812 21:08:31.809057 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.809060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.809138 (+ 78us) server_negotiation.cc:407] Connection header received
0812 21:08:31.809239 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.809240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.809251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.809309 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:31.809503 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.810258 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.811819 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.812418 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.812476 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.812485 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.812672 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.812715 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:31.812867 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.808915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:46375 (local address 127.11.207.130:33981)
0812 21:08:31.809131 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.809140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.809190 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.809334 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.809335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.809337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.809421 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.809422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.810333 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.810335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.811705 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.811710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.812490 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.812491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.812590 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.812600 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.812664 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.812753 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":59,"thread_start_us":66,"threads_started":1}
I20260812 21:08:31.812884 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.808911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:41977 (local address 127.11.207.130:43223)
0812 21:08:31.809055 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.809066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.809136 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.809352 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.809352 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.809354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:31.809420 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.809422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.810350 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.810350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.811737 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.811739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.812491 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.812492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.812590 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.812597 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.812649 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.812748 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":85,"thread_start_us":73,"threads_started":1}
I20260812 21:08:31.813271 14505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10af424982bc44bea3c9ae4c3ed61632" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996acf083aa447adb2bab1f81c06ebf4" is_pre_election: true
I20260812 21:08:31.813361 14505 raft_consensus.cc:2454] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 0.
I20260812 21:08:31.813335 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10af424982bc44bea3c9ae4c3ed61632" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ce6a6d7b1744988ba60ee983de8724" is_pre_election: true
I20260812 21:08:31.813510 15011 raft_consensus.cc:2454] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 0.
I20260812 21:08:31.813550 14631 leader_election.cc:302] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f1abd6392a4b8480a838641f5466d1, 996acf083aa447adb2bab1f81c06ebf4; no voters:
I20260812 21:08:31.813689 15169 raft_consensus.cc:2781] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:31.813745 15169 raft_consensus.cc:482] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:31.813766 15169 raft_consensus.cc:3037] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:31.814419 15169 raft_consensus.cc:504] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.814535 15169 leader_election.cc:288] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 election: Requested vote from peers 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977), b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375)
I20260812 21:08:31.814682 14505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10af424982bc44bea3c9ae4c3ed61632" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996acf083aa447adb2bab1f81c06ebf4"
I20260812 21:08:31.814707 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10af424982bc44bea3c9ae4c3ed61632" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ce6a6d7b1744988ba60ee983de8724"
I20260812 21:08:31.814752 14505 raft_consensus.cc:3037] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:31.814769 15011 raft_consensus.cc:3037] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:31.815500 14505 raft_consensus.cc:2454] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 1.
I20260812 21:08:31.815559 15011 raft_consensus.cc:2454] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 1.
I20260812 21:08:31.815713 14630 leader_election.cc:302] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f1abd6392a4b8480a838641f5466d1, b9ce6a6d7b1744988ba60ee983de8724; no voters:
I20260812 21:08:31.815804 15169 raft_consensus.cc:2781] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:31.815862 15169 raft_consensus.cc:686] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 1 LEADER]: Becoming Leader. State: Replica: 33f1abd6392a4b8480a838641f5466d1, State: Running, Role: LEADER
I20260812 21:08:31.815925 15169 consensus_queue.cc:237] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.816640 14216 catalog_manager.cc:5495] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } health_report { overall_health: UNKNOWN } } }
W20260812 21:08:31.836505 14595 tablet.cc:2369] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 21:08:31 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568910, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:31.903415 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.897074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43825 (local address 127.0.0.1:42454)
0812 21:08:31.897232 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:31.897241 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:31.897309 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:31.897439 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:31.897440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:31.897747 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:31.897795 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.897797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.898408 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.898409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.898937 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:31.898938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.899295 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.899296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:31.899339 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.899809 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:31.899814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:31.901274 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:31.902487 (+ 1213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.902495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.902497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.902688 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.902891 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:31.902892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:31.902892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:31.902944 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:31.903132 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:31.903137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:31.903179 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 21:08:31.903228 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:08:31.903318 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260812 21:08:31.903465 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:31.897134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.11.207.190:43825)
0812 21:08:31.897234 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:31.897237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:31.897303 (+ 66us) server_negotiation.cc:407] Connection header received
0812 21:08:31.897365 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:31.897366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:31.897376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:31.897428 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:31.897857 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.898353 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.899025 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:31.899230 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:31.899287 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:31.901348 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:31.901349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:31.901350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:31.901355 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:31.902421 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.902773 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.902774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.902774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.902835 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:31.902985 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:31.902986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:31.902986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:31.903088 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:31.903135 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:31.903256 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 21:08:31.903369 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":74}
I20260812 21:08:31.905768 14216 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:42454:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 21:08:31.906071 14216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:31.907658 14720 tablet_service.cc:1467] Processing CreateTablet for tablet b7b9815d94a742e3b7049d2afcd5aa41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96fea41d3c784620937aa960fa1717f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:08:31.907752 14978 tablet_service.cc:1467] Processing CreateTablet for tablet b7b9815d94a742e3b7049d2afcd5aa41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96fea41d3c784620937aa960fa1717f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:08:31.907876 14720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b9815d94a742e3b7049d2afcd5aa41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.907943 14978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b9815d94a742e3b7049d2afcd5aa41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.909109 15137 tablet_bootstrap.cc:492] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: Bootstrap starting.
I20260812 21:08:31.909214 15136 tablet_bootstrap.cc:492] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: Bootstrap starting.
I20260812 21:08:31.909539 15136 tablet_bootstrap.cc:654] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.909754 15137 tablet_bootstrap.cc:654] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.907658 14464 tablet_service.cc:1467] Processing CreateTablet for tablet b7b9815d94a742e3b7049d2afcd5aa41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96fea41d3c784620937aa960fa1717f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:08:31.910020 14464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b9815d94a742e3b7049d2afcd5aa41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:31.910156 15136 tablet_bootstrap.cc:492] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: No bootstrap required, opened a new log
I20260812 21:08:31.910197 15136 ts_tablet_manager.cc:1397] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:31.910303 15136 raft_consensus.cc:348] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.910385 15136 raft_consensus.cc:374] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.910403 15136 raft_consensus.cc:729] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ce6a6d7b1744988ba60ee983de8724, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.910444 15136 consensus_queue.cc:260] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.910519 15136 ts_tablet_manager.cc:1428] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:31.910625 15147 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:31.910660 15137 tablet_bootstrap.cc:492] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: No bootstrap required, opened a new log
I20260812 21:08:31.910693 15147 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:31.910704 15137 ts_tablet_manager.cc:1397] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:31.910933 15137 raft_consensus.cc:348] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.911011 15137 raft_consensus.cc:374] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.911029 15137 raft_consensus.cc:729] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f1abd6392a4b8480a838641f5466d1, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.911068 15137 consensus_queue.cc:260] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.911149 15137 ts_tablet_manager.cc:1428] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:31.911257 15169 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:31.911335 15169 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:31.911573 15144 tablet_bootstrap.cc:492] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: Bootstrap starting.
I20260812 21:08:31.912096 15144 tablet_bootstrap.cc:654] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:31.912817 15144 tablet_bootstrap.cc:492] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: No bootstrap required, opened a new log
I20260812 21:08:31.912878 15144 ts_tablet_manager.cc:1397] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:31.913076 15144 raft_consensus.cc:348] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.913139 15144 raft_consensus.cc:374] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:31.913152 15144 raft_consensus.cc:729] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996acf083aa447adb2bab1f81c06ebf4, State: Initialized, Role: FOLLOWER
I20260812 21:08:31.913199 15144 consensus_queue.cc:260] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:31.913290 15144 ts_tablet_manager.cc:1428] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:31.913429 15149 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:31.913511 15149 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:32.243193 15169 raft_consensus.cc:482] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:32.243270 15169 raft_consensus.cc:504] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:32.243415 15169 leader_election.cc:288] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977), b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375)
I20260812 21:08:32.243659 14505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b9815d94a742e3b7049d2afcd5aa41" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996acf083aa447adb2bab1f81c06ebf4" is_pre_election: true
I20260812 21:08:32.243659 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b9815d94a742e3b7049d2afcd5aa41" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ce6a6d7b1744988ba60ee983de8724" is_pre_election: true
I20260812 21:08:32.243765 14505 raft_consensus.cc:2454] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 0.
I20260812 21:08:32.243767 15011 raft_consensus.cc:2454] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 0.
I20260812 21:08:32.243908 14630 leader_election.cc:302] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f1abd6392a4b8480a838641f5466d1, b9ce6a6d7b1744988ba60ee983de8724; no voters:
I20260812 21:08:32.244031 15169 raft_consensus.cc:2781] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:32.244078 15169 raft_consensus.cc:482] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:32.244096 15169 raft_consensus.cc:3037] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:32.244844 15169 raft_consensus.cc:504] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:32.244969 15169 leader_election.cc:288] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 election: Requested vote from peers 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977), b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375)
I20260812 21:08:32.245101 14505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b9815d94a742e3b7049d2afcd5aa41" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996acf083aa447adb2bab1f81c06ebf4"
I20260812 21:08:32.245169 14505 raft_consensus.cc:3037] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:32.245718 14505 raft_consensus.cc:2454] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 1.
I20260812 21:08:32.245842 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7b9815d94a742e3b7049d2afcd5aa41" candidate_uuid: "33f1abd6392a4b8480a838641f5466d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ce6a6d7b1744988ba60ee983de8724"
I20260812 21:08:32.245877 14631 leader_election.cc:302] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f1abd6392a4b8480a838641f5466d1, 996acf083aa447adb2bab1f81c06ebf4; no voters:
I20260812 21:08:32.245923 15011 raft_consensus.cc:3037] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:32.245973 15169 raft_consensus.cc:2781] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:32.246028 15169 raft_consensus.cc:686] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [term 1 LEADER]: Becoming Leader. State: Replica: 33f1abd6392a4b8480a838641f5466d1, State: Running, Role: LEADER
I20260812 21:08:32.246078 15169 consensus_queue.cc:237] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } }
I20260812 21:08:32.246315 15169 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: TxnStatusTablet state changed. Reason: New leader 33f1abd6392a4b8480a838641f5466d1. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.246373 15169 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:08:32.246549 15208 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 21:08:32.246708 15208 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 21:08:32.246758 15011 raft_consensus.cc:2454] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33f1abd6392a4b8480a838641f5466d1 in term 1.
I20260812 21:08:32.246714 14216 catalog_manager.cc:5495] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } health_report { overall_health: UNKNOWN } } }
I20260812 21:08:32.265139 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.262245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44445 (local address 127.0.0.1:50928)
0812 21:08:32.262315 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:32.262327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:32.262372 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:32.262726 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:32.262727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:32.262728 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:32.262784 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:32.262786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.263639 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.263640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:32.264231 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:32.264232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.264632 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.264633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:32.264688 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.264705 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:32.264949 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:32.264959 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 21:08:32.265010 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:08:32.265065 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 21:08:32.265162 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.262308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50928 (local address 127.11.207.130:44445)
0812 21:08:32.262583 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:32.262585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:32.262592 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:32.262625 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:32.262626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:32.262643 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:32.262711 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:32.262897 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.263560 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.264305 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.264558 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.264612 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.264791 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:32.264894 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:32.264958 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:32.265015 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:08:32.265040 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":53,"threads_started":1}
I20260812 21:08:32.266475 14505 raft_consensus.cc:1261] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Refusing update from remote peer 33f1abd6392a4b8480a838641f5466d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:32.266543 15011 raft_consensus.cc:1261] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Refusing update from remote peer 33f1abd6392a4b8480a838641f5466d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:32.266839 15145 consensus_queue.cc:1035] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:32.266965 15169 consensus_queue.cc:1035] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:32.267668 15149 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: TxnStatusTablet state changed. Reason: New leader 33f1abd6392a4b8480a838641f5466d1. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.267678 15147 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: TxnStatusTablet state changed. Reason: New leader 33f1abd6392a4b8480a838641f5466d1. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.267771 15147 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:32.267771 15149 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:32.268754 15169 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.268841 15169 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:08:32.268932 15149 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.268965 15215 mvcc.cc:205] Tried to move back new op lower bound from 7317786264641404928 to 7317786264560353280. Current Snapshot: MvccSnapshot[applied={T|T < 7317786264641404928}]
I20260812 21:08:32.269006 15149 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 996acf083aa447adb2bab1f81c06ebf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:32.269341 15231 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.269464 15231 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P 33f1abd6392a4b8480a838641f5466d1: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:08:32.269637 15147 tablet_replica.cc:439] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33f1abd6392a4b8480a838641f5466d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 } } peers { permanent_uuid: "33f1abd6392a4b8480a838641f5466d1" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44445 } } peers { permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 } } }
I20260812 21:08:32.269728 15147 tablet_replica.cc:442] T b7b9815d94a742e3b7049d2afcd5aa41 P b9ce6a6d7b1744988ba60ee983de8724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:32.273886 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.271396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.11.207.130:44445)
0812 21:08:32.271433 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:32.271435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:32.271473 (+ 38us) server_negotiation.cc:407] Connection header received
0812 21:08:32.271545 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:32.271546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:32.271562 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:32.271604 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:32.271719 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.272167 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.273279 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.273489 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.273539 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.273673 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:32.273698 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:32.273729 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:32.273793 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 21:08:32.273810 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:08:32.279824 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.276546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:34467 (local address 127.11.207.130:44445)
0812 21:08:32.276585 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:32.276587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:32.276677 (+ 90us) server_negotiation.cc:407] Connection header received
0812 21:08:32.276743 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:32.276743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:32.276750 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:32.276791 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:32.276914 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.277339 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.278656 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.278831 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.278879 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.279608 (+ 729us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:32.279635 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:32.279671 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:32.279739 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 21:08:32.279754 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:32.279938 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.276479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44445 (local address 127.11.207.130:34467)
0812 21:08:32.276623 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:32.276633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:32.276691 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:32.276795 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:32.276796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:32.276807 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:32.276863 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:32.276864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.277396 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.277397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:32.278580 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:32.278583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.278892 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.278892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:32.279509 (+ 617us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.279527 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:32.279686 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:32.279688 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:32.279736 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:08:32.279782 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260812 21:08:32.282224 15011 raft_consensus.cc:1261] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Refusing update from remote peer 33f1abd6392a4b8480a838641f5466d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:32.282208 14505 raft_consensus.cc:1261] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Refusing update from remote peer 33f1abd6392a4b8480a838641f5466d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:32.282487 15145 consensus_queue.cc:1035] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ce6a6d7b1744988ba60ee983de8724" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:32.282626 15231 consensus_queue.cc:1035] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "996acf083aa447adb2bab1f81c06ebf4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:32.297189 14216 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:32.297292 14216 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:32.297907 14216 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9b9e0cd23164ae8a97802eecc039d91: Sending DeleteTablet for 3 replicas of tablet 10af424982bc44bea3c9ae4c3ed61632
I20260812 21:08:32.298177 14978 tablet_service.cc:1514] Processing DeleteTablet for tablet 10af424982bc44bea3c9ae4c3ed61632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:32 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:52814
I20260812 21:08:32.298151 14464 tablet_service.cc:1514] Processing DeleteTablet for tablet 10af424982bc44bea3c9ae4c3ed61632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:32 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:36730
I20260812 21:08:32.298169 14720 tablet_service.cc:1514] Processing DeleteTablet for tablet 10af424982bc44bea3c9ae4c3ed61632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:32 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:50924
I20260812 21:08:32.298516 15252 tablet_replica.cc:330] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: stopping tablet replica
I20260812 21:08:32.298530 15254 tablet_replica.cc:330] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: stopping tablet replica
I20260812 21:08:32.298588 15252 raft_consensus.cc:2229] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:32.298632 15252 raft_consensus.cc:2258] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:32.298641 15254 raft_consensus.cc:2229] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:32.298733 15253 tablet_replica.cc:330] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: stopping tablet replica
I20260812 21:08:32.298786 15254 raft_consensus.cc:2258] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:32.298794 15253 raft_consensus.cc:2229] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:32.298830 15253 raft_consensus.cc:2258] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:32.299067 15253 ts_tablet_manager.cc:1905] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:32.299225 15254 ts_tablet_manager.cc:1905] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:32.300848 15253 ts_tablet_manager.cc:1918] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:32.300913 15253 log.cc:1196] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/10af424982bc44bea3c9ae4c3ed61632
I20260812 21:08:32.301136 15253 ts_tablet_manager.cc:1939] T 10af424982bc44bea3c9ae4c3ed61632 P b9ce6a6d7b1744988ba60ee983de8724: Deleting consensus metadata
I20260812 21:08:32.301307 15254 ts_tablet_manager.cc:1918] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:32.301406 15254 log.cc:1196] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/10af424982bc44bea3c9ae4c3ed61632
I20260812 21:08:32.301479 14195 catalog_manager.cc:4841] TS b9ce6a6d7b1744988ba60ee983de8724 (127.11.207.131:46375): tablet 10af424982bc44bea3c9ae4c3ed61632 (table test-table [id=1e7f9dfd78c046868369b251c7f08204]) successfully deleted
I20260812 21:08:32.301713 15252 ts_tablet_manager.cc:1905] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:32.301893 15254 ts_tablet_manager.cc:1939] T 10af424982bc44bea3c9ae4c3ed61632 P 33f1abd6392a4b8480a838641f5466d1: Deleting consensus metadata
I20260812 21:08:32.302248 14198 catalog_manager.cc:4841] TS 33f1abd6392a4b8480a838641f5466d1 (127.11.207.130:44445): tablet 10af424982bc44bea3c9ae4c3ed61632 (table test-table [id=1e7f9dfd78c046868369b251c7f08204]) successfully deleted
I20260812 21:08:32.303424 15252 ts_tablet_manager.cc:1918] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:32.303490 15252 log.cc:1196] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/10af424982bc44bea3c9ae4c3ed61632
I20260812 21:08:32.303702 15252 ts_tablet_manager.cc:1939] T 10af424982bc44bea3c9ae4c3ed61632 P 996acf083aa447adb2bab1f81c06ebf4: Deleting consensus metadata
I20260812 21:08:32.304041 14196 catalog_manager.cc:4841] TS 996acf083aa447adb2bab1f81c06ebf4 (127.11.207.129:41977): tablet 10af424982bc44bea3c9ae4c3ed61632 (table test-table [id=1e7f9dfd78c046868369b251c7f08204]) successfully deleted
Aug 12 21:08:32 dist-test-slave-lkx2 krb5kdc[14136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568911, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:32.308754 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.299540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36744 (local address 127.11.207.129:41977)
0812 21:08:32.299579 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:32.299582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:32.299822 (+ 240us) server_negotiation.cc:407] Connection header received
0812 21:08:32.299916 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:32.299917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:32.299929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:32.299957 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:32.300789 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.301492 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.302469 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.302681 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.302736 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.306061 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:32.306072 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:32.306072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:32.306079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:32.307614 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:32.307996 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:32.307999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:32.308000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:32.308065 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:32.308262 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:32.308263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:32.308263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:32.308427 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:32.308542 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:32.308625 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 21:08:32.308691 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:08:32.309166 14583 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36744
I20260812 21:08:32.316113 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:32.310155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.11.207.190:43825)
0812 21:08:32.310223 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:32.310225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:32.310321 (+ 96us) server_negotiation.cc:407] Connection header received
0812 21:08:32.310388 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:32.310389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:32.310401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:32.310461 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:32.310893 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.311692 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.312322 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:32.312653 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:32.312722 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:32.313844 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:32.313846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:32.313846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:32.313853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:32.315160 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:32.315461 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:32.315462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:32.315463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:32.315526 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:32.315658 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:32.315658 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:32.315659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:32.315781 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:32.315868 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:32.315943 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 21:08:32.316022 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 21:08:32.316378 14216 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42460
I20260812 21:08:32.317026 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14340
I20260812 21:08:32.372560 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14605
I20260812 21:08:32.388728 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14857
I20260812 21:08:32.405532 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14163
2026-08-12T21:08:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2990 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:32 dist-test-slave-lkx2 krb5kdc[15266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:32 dist-test-slave-lkx2 krb5kdc[15266](info): set up 2 sockets
Aug 12 21:08:32 dist-test-slave-lkx2 krb5kdc[15266](info): commencing operation
krb5kdc: starting...
W20260812 21:08:33.614923 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:33 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:33.736754 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41623
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:33.824084 15292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:33.824319 15292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:33.824378 15292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:33.825587 15292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:33.825631 15292 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 21:08:33.825649 15292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:33.825666 15292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:33.825682 15292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:33.825698 15292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:33.827270 15292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41623
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:33.827553 15292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:33.827795 15292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:33.832703 15305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:33.832723 15302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:33.832703 15303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:33.832998 15292 server_base.cc:1034] running on GCE node
I20260812 21:08:33.833257 15292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:33.833546 15292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:33.834726 15292 hybrid_clock.cc:648] HybridClock initialized: now 1786568913834717 us; error 51 us; skew 500 ppm
Aug 12 21:08:33 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568913, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:33.837826 15292 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:33.838310 15292 webserver.cc:460] Webserver started at http://127.11.207.190:33975/ using document root <none> and password file <none>
I20260812 21:08:33.838529 15292 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:33.838580 15292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:33.838734 15292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:33.840080 15292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7"
format_stamp: "Formatted at 2026-08-12 21:08:33 on dist-test-slave-lkx2"
server_key: "1eef8de2f2cd7f814bc9c3bc748395be"
server_key_iv: "400ee81dba1d737e1e79c7e29916d958"
server_key_version: "encryptionkey@0"
I20260812 21:08:33.840480 15292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7"
format_stamp: "Formatted at 2026-08-12 21:08:33 on dist-test-slave-lkx2"
server_key: "1eef8de2f2cd7f814bc9c3bc748395be"
server_key_iv: "400ee81dba1d737e1e79c7e29916d958"
server_key_version: "encryptionkey@0"
I20260812 21:08:33.841842 15292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:33.842581 15318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:33.842736 15292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:33.842828 15292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7"
format_stamp: "Formatted at 2026-08-12 21:08:33 on dist-test-slave-lkx2"
server_key: "1eef8de2f2cd7f814bc9c3bc748395be"
server_key_iv: "400ee81dba1d737e1e79c7e29916d958"
server_key_version: "encryptionkey@0"
I20260812 21:08:33.842896 15292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:33.858665 15292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:33.861059 15292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:33.861220 15292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:33.866004 15292 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41623
I20260812 21:08:33.866042 15421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41623 every 8 connection(s)
I20260812 21:08:33.866406 15292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:33.867118 15422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:33.869714 15422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Bootstrap starting.
I20260812 21:08:33.870440 15422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:33.870839 15422 log.cc:824] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:33.871546 15422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: No bootstrap required, opened a new log
I20260812 21:08:33.872491 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15292
I20260812 21:08:33.872577 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:33.872764 12094 external_mini_cluster.cc:1428] Setting key 34c5a7c8d8e755ab61e3e9965ea9bf94
I20260812 21:08:33.874260 15422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41623 } }
I20260812 21:08:33.874411 15422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:33.874445 15422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ed7e8c8f46c4f79b0ee27e7e386b4f7, State: Initialized, Role: FOLLOWER
I20260812 21:08:33.874570 15422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41623 } }
I20260812 21:08:33.874655 15422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:33.874735 15422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:33.874789 15422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:33.875490 15422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41623 } }
I20260812 21:08:33.875584 15422 leader_election.cc:302] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ed7e8c8f46c4f79b0ee27e7e386b4f7; no voters:
I20260812 21:08:33.875775 15422 leader_election.cc:288] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:33.875816 15431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:33.875972 15431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [term 1 LEADER]: Becoming Leader. State: Replica: 7ed7e8c8f46c4f79b0ee27e7e386b4f7, State: Running, Role: LEADER
I20260812 21:08:33.876050 15422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:33.876075 15431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41623 } }
I20260812 21:08:33.876483 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41623 } } }
I20260812 21:08:33.876593 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:33.876770 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ed7e8c8f46c4f79b0ee27e7e386b4f7. Latest consensus state: current_term: 1 leader_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed7e8c8f46c4f79b0ee27e7e386b4f7" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41623 } } }
I20260812 21:08:33.876837 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7 [sys.catalog]: This master's current role is: LEADER
W20260812 21:08:33.877911 15462 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:33.877977 15462 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:33.878031 15447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:33.878211 15447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:33.879720 15447 catalog_manager.cc:1347] Generated new cluster ID: d93bce6e59c540c7b936956fbc7e936c
I20260812 21:08:33.879771 15447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:33 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:33.884729 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:33.873359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59302 (local address 127.11.207.190:41623)
0812 21:08:33.873582 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:33.873590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:33.873618 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:08:33.874245 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:33.874255 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:33.874316 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:33.874465 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:33.875210 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:33.876461 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:33.877504 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:33.878138 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:33.878247 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:33.881007 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:33.881011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:33.881011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:33.881023 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:33.882899 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:33.883294 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:33.883296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:33.883297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:33.883421 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:33.883610 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:33.883610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:33.883611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:33.883890 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:33.884040 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:33.884313 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 21:08:33.884519 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
I20260812 21:08:33.886207 15447 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:33.886689 15447 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:33.897807 15447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Generated new TSK 0
I20260812 21:08:33.898015 15447 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:33.929560 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41623
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:34.013011 15473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:34.013252 15473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:34.013290 15473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:34.014537 15473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:34.014585 15473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:34.014613 15473 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 21:08:34.014627 15473 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:08:34.014652 15473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:34.016179 15473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:41623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:34.016458 15473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:34.016680 15473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:34.021692 15486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:34.021708 15480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:34.021708 15483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:34.022043 15473 server_base.cc:1034] running on GCE node
I20260812 21:08:34.022238 15473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:34.022482 15473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:34.023625 15473 hybrid_clock.cc:648] HybridClock initialized: now 1786568914023609 us; error 28 us; skew 500 ppm
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:34.026119 15473 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:34.026582 15473 webserver.cc:460] Webserver started at http://127.11.207.129:44321/ using document root <none> and password file <none>
I20260812 21:08:34.026815 15473 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:34.026868 15473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:34.026997 15473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:34.028079 15473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "58ec1042b25f40dc88d7e1ba85767191"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "faec19c2a289c176efbd163832c297a5"
server_key_iv: "cfbc11b0b7366ec82f9616c714d65bcd"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.028534 15473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "58ec1042b25f40dc88d7e1ba85767191"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "faec19c2a289c176efbd163832c297a5"
server_key_iv: "cfbc11b0b7366ec82f9616c714d65bcd"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.029817 15473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:34.030488 15499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.030627 15473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:34.030717 15473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "58ec1042b25f40dc88d7e1ba85767191"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "faec19c2a289c176efbd163832c297a5"
server_key_iv: "cfbc11b0b7366ec82f9616c714d65bcd"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.030783 15473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:34.067814 15473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:34.070194 15473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:34.070325 15473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:34.070916 15473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:34.070962 15473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.070986 15473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:34.070998 15473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:34.078019 15473 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:44237
I20260812 21:08:34.078102 15707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:44237 every 8 connection(s)
I20260812 21:08:34.078423 15473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:34.081029 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.071442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:47687 (local address 127.11.207.190:41623)
0812 21:08:34.071498 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.071501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.071792 (+ 291us) server_negotiation.cc:407] Connection header received
0812 21:08:34.072386 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.072389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.072404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.072472 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.073334 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.074106 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.075568 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.075885 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.075964 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.078438 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.078439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.078439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.078452 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.079756 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.080100 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.080101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.080101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.080150 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.080392 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.080393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.080393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.080521 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.080575 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.080870 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.080963 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:08:34.081101 15510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.071454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.11.207.129:47687)
0812 21:08:34.071645 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.071693 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.072224 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.072543 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.072548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.072900 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.073218 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.073225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.074196 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.074197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.075477 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.075481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.075977 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.075978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.076115 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.076653 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.076706 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.078343 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.079822 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.079829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.079831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.080044 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.080197 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.080197 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.080198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.080298 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.080576 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.080582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.080712 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.080802 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.080927 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260812 21:08:34.081719 15710 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41623
I20260812 21:08:34.081802 15710 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:34.082008 15710 heartbeater.cc:511] Master 127.11.207.190:41623 requested a full tablet report, sending...
I20260812 21:08:34.082633 15346 ts_manager.cc:194] Registered new tserver with Master: 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237)
I20260812 21:08:34.083410 15346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:47687
I20260812 21:08:34.086097 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15473
I20260812 21:08:34.086210 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:34.086421 12094 external_mini_cluster.cc:1428] Setting key d0c633e888a3eb5cc5973c1218e8bd8f
I20260812 21:08:34.088603 15724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.082760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.11.207.129:35045)
0812 21:08:34.082955 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.082966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.083023 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.083132 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.083133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.083280 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.083338 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.083340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.084110 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.084111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.084717 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.084720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.085083 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.085084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.085139 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.085541 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.085545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.085858 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.087613 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.087614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.087614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.087774 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.087969 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.087970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.087970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.088017 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.088310 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.088311 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.088391 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.088447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.088530 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":133,"threads_started":1}
I20260812 21:08:34.088666 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.082806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:35045 (local address 127.11.207.190:41623)
0812 21:08:34.082844 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.082846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.083026 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:08:34.083062 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.083063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.083072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.083129 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.083395 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.084012 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.084818 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.085014 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.085084 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.085942 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.085943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.085944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.085952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.087534 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.087838 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.087839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.087839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.087909 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.088112 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.088113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.088113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.088252 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.088322 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.088489 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.088577 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:34.115155 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41623
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:34.200691 15728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:34.200917 15728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:34.200941 15728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:34.202060 15728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:34.202095 15728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:34.202114 15728 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 21:08:34.202124 15728 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:08:34.202139 15728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:34.203542 15728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:41623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:34.203742 15728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:34.203923 15728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:34.208863 15736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:34.208863 15739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:34.208911 15728 server_base.cc:1034] running on GCE node
W20260812 21:08:34.208907 15737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:34.209234 15728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:34.209472 15728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:34.210638 15728 hybrid_clock.cc:648] HybridClock initialized: now 1786568914210621 us; error 29 us; skew 500 ppm
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:34.213231 15728 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:34.213683 15728 webserver.cc:460] Webserver started at http://127.11.207.130:46289/ using document root <none> and password file <none>
I20260812 21:08:34.213899 15728 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:34.213949 15728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:34.214138 15728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:34.215217 15728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "ec95c187506d430fb17be4d6da345a80"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "4eb5014988f55bebce650a8f78cffa28"
server_key_iv: "fac3326d6832272fdd62afa23076490b"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.215556 15728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "ec95c187506d430fb17be4d6da345a80"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "4eb5014988f55bebce650a8f78cffa28"
server_key_iv: "fac3326d6832272fdd62afa23076490b"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.216934 15728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:34.217656 15751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.217797 15728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:34.217900 15728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "ec95c187506d430fb17be4d6da345a80"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "4eb5014988f55bebce650a8f78cffa28"
server_key_iv: "fac3326d6832272fdd62afa23076490b"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.217962 15728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:34.229943 15728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:34.231796 15728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:34.231936 15728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:34.232481 15728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:34.232524 15728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.232560 15728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:34.232584 15728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:34.239467 15728 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:44941
I20260812 21:08:34.239554 15952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:44941 every 8 connection(s)
I20260812 21:08:34.239892 15728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:34.241369 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15728
I20260812 21:08:34.241472 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:34.241685 12094 external_mini_cluster.cc:1428] Setting key 649f2b63a2df71c1e44f20a552e5d002
I20260812 21:08:34.242285 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.232900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:50697 (local address 127.11.207.190:41623)
0812 21:08:34.232940 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.232942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.233168 (+ 226us) server_negotiation.cc:407] Connection header received
0812 21:08:34.233757 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.233759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.233772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.233822 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.234489 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.235087 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.236111 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.236314 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.236399 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.239540 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.239542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.239542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.239551 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.240959 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.241317 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.241318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.241318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.241381 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.241635 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.241636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.241636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.241777 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.241846 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.242132 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.242205 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:08:34.242539 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.232896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.11.207.130:50697)
0812 21:08:34.233042 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.233078 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.233640 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.233893 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.233897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.234178 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.234402 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.234407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.235181 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.235182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.236025 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.236027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.236407 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.236408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.236547 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.237006 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.237018 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.239428 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.241029 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.241034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.241036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.241246 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.241448 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.241449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.241449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.241580 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.241835 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.241841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.242005 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.242120 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.242287 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260812 21:08:34.242993 15954 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41623
I20260812 21:08:34.243072 15954 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:34.243254 15954 heartbeater.cc:511] Master 127.11.207.190:41623 requested a full tablet report, sending...
I20260812 21:08:34.243615 15347 ts_manager.cc:194] Registered new tserver with Master: ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941)
I20260812 21:08:34.244093 15347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:50697
I20260812 21:08:34.249884 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.244241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:45667 (local address 127.11.207.190:41623)
0812 21:08:34.244285 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.244287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.244467 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:08:34.244557 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.244558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.244568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.244595 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.244986 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.245608 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.246538 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.246713 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.246759 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.247680 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.247681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.247682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.247688 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.248890 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.249239 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.249240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.249240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.249282 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.249436 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.249436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.249436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.249562 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.249609 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.249764 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.249820 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:34.249948 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.244033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.11.207.130:45667)
0812 21:08:34.244399 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.244411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.244492 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.244650 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.244651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.244836 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.244905 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.244907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.245699 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.245700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.246465 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.246469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.246784 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.246785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.246863 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.247237 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.247243 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.247609 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.248966 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.248966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.248967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.249174 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.249320 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.249321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.249321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.249386 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.249614 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.249615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.249664 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.249745 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.249848 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:34.268733 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41623
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:34.351783 15967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:34.352065 15967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:34.352101 15967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:34.353518 15967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:34.353569 15967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:34.353590 15967 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 21:08:34.353600 15967 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:08:34.353616 15967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:34.355113 15967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:41623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:34.355324 15967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:34.355511 15967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:34.360435 15974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:34.360574 15967 server_base.cc:1034] running on GCE node
W20260812 21:08:34.360442 15977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:34.360548 15975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:34.360874 15967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:34.361133 15967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:34.362310 15967 hybrid_clock.cc:648] HybridClock initialized: now 1786568914362294 us; error 28 us; skew 500 ppm
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:34.364876 15967 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:34.365314 15967 webserver.cc:460] Webserver started at http://127.11.207.131:39565/ using document root <none> and password file <none>
I20260812 21:08:34.365505 15967 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:34.365545 15967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:34.365639 15967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:34.366657 15967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "dd67a13d15d048e18679ceb7e11c1ce9"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "80717bd66ff768451114fedc819e3e7b"
server_key_iv: "50f8ee39aeeb834790d683e3e14792e3"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.366966 15967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "dd67a13d15d048e18679ceb7e11c1ce9"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "80717bd66ff768451114fedc819e3e7b"
server_key_iv: "50f8ee39aeeb834790d683e3e14792e3"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.368301 15967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:34.369077 15992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.369246 15967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:34.369334 15967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "dd67a13d15d048e18679ceb7e11c1ce9"
format_stamp: "Formatted at 2026-08-12 21:08:34 on dist-test-slave-lkx2"
server_key: "80717bd66ff768451114fedc819e3e7b"
server_key_iv: "50f8ee39aeeb834790d683e3e14792e3"
server_key_version: "encryptionkey@0"
I20260812 21:08:34.369392 15967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:34.380023 15967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:34.382212 15967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:34.382340 15967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:34.382824 15967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:34.382875 15967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.382900 15967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:34.382915 15967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:34.390049 15967 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:33413
I20260812 21:08:34.390085 16174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:33413 every 8 connection(s)
I20260812 21:08:34.390461 15967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:34.392752 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.383315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37923 (local address 127.11.207.190:41623)
0812 21:08:34.383381 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.383383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.383560 (+ 177us) server_negotiation.cc:407] Connection header received
0812 21:08:34.384073 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.384074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.384087 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.384142 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.385222 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.385830 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.387306 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.387556 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.387609 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.389968 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.389970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.389970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.389979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.391452 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.391823 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.391824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.391824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.391882 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.392095 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.392096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.392096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.392212 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.392268 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.392571 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.392656 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:08:34.392897 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.383290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.11.207.131:37923)
0812 21:08:34.383447 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.383488 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.383979 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.384261 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.384266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.384656 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.385115 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.385124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.385913 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.385914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.387203 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.387206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.387627 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.387628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.387773 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.388279 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.388290 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.389885 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.391522 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.391529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.391531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.391754 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.391933 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.391934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.391935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.392041 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.392265 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.392272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.392450 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.392540 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.392694 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260812 21:08:34.393318 16175 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41623
I20260812 21:08:34.393422 16175 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:34.393625 16175 heartbeater.cc:511] Master 127.11.207.190:41623 requested a full tablet report, sending...
I20260812 21:08:34.393738 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15967
I20260812 21:08:34.393823 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:34.393967 15346 ts_manager.cc:194] Registered new tserver with Master: dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:34.394023 12094 external_mini_cluster.cc:1428] Setting key aa5b51fc45dd426f3b3ed4f6abb41451
I20260812 21:08:34.394403 15346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:37923
I20260812 21:08:34.395820 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:08:34.399677 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.394333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:45233 (local address 127.11.207.190:41623)
0812 21:08:34.394377 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.394379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.394429 (+ 50us) server_negotiation.cc:407] Connection header received
0812 21:08:34.394496 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.394497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.394506 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.394555 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.394893 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.395353 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.396185 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.396416 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.396471 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.397230 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.397231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.397231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.397238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.398645 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.398971 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.398971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.398972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.399030 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.399185 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.399186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.399186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.399308 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.399370 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.399504 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.399575 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:34.399677 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.394262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.11.207.131:45233)
0812 21:08:34.394379 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.394389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.394442 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.394560 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.394561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.394743 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.394829 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.394831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.395434 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.395435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.396107 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.396110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.396470 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.396471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.396550 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.396897 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.396900 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.397150 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.398744 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.398745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.398745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.398913 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.399086 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.399087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.399087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.399132 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.399363 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.399365 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.399413 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.399476 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.399577 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:34.419924 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.413258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.11.207.190:41623)
0812 21:08:34.413338 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.413340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.413348 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:34.413375 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.413376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.413396 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.413447 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.413971 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.414481 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.415241 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.415434 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.415496 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.417632 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.417633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.417634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.417642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.418781 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.419123 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.419124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.419124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.419195 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.419421 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.419422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.419422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.419536 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.419628 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.419746 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.419828 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:08:34.421590 15346 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:34.423255 15346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:34.429682 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.425616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44941 (local address 127.0.0.1:32806)
0812 21:08:34.425972 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.425980 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.426012 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.426255 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.426259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.426261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.426342 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.426344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.427429 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.427430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.428551 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.428554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.429391 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.429392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.429471 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.429485 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.429550 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.429606 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":55,"threads_started":1}
I20260812 21:08:34.429978 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.425925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.11.207.130:44941)
0812 21:08:34.426106 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.426111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.426121 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:34.426154 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.426155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.426168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.426235 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.426404 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.427356 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.428659 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.429320 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.429385 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.429447 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.429867 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.429906 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260812 21:08:34.429970 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.425749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.11.207.129:44237)
0812 21:08:34.426131 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.426134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.426144 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:34.426184 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.426185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.426201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.426275 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.426530 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.427632 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.428826 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.429460 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.429732 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.429793 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.429819 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.429869 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":68,"threads_started":1}
I20260812 21:08:34.429790 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.425614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:44237 (local address 127.0.0.1:43284)
0812 21:08:34.425754 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.425797 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.425863 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.426295 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.426296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.426299 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.426361 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.426363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.427726 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.427728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.428757 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.428759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.429554 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.429555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.429614 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.429619 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.429648 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.429686 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":56,"thread_start_us":90,"threads_started":1}
I20260812 21:08:34.430341 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.425856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.0.0.1:51942)
0812 21:08:34.426378 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.426388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.426440 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.426668 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.426669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.426672 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.426718 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.426720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.427946 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.427947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.429007 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.429010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.430141 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.430143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.430201 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.430206 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.430239 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.430275 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":75,"threads_started":1}
I20260812 21:08:34.430714 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.425941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.11.207.131:33413)
0812 21:08:34.426117 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.426121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.426450 (+ 329us) server_negotiation.cc:407] Connection header received
0812 21:08:34.426506 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.426509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.426525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.426598 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.426804 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.427865 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.429089 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.429987 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.430494 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.430556 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.430580 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.430621 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260812 21:08:34.430858 15844 tablet_service.cc:1467] Processing CreateTablet for tablet abec541bd00c4722a372edeae5d93342 (DEFAULT_TABLE table=test-table [id=16146069478f4015963e255a6c2c6955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:34.430953 15580 tablet_service.cc:1467] Processing CreateTablet for tablet abec541bd00c4722a372edeae5d93342 (DEFAULT_TABLE table=test-table [id=16146069478f4015963e255a6c2c6955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:34.431254 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abec541bd00c4722a372edeae5d93342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.431262 15844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abec541bd00c4722a372edeae5d93342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.431277 16062 tablet_service.cc:1467] Processing CreateTablet for tablet abec541bd00c4722a372edeae5d93342 (DEFAULT_TABLE table=test-table [id=16146069478f4015963e255a6c2c6955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:34.431594 16062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abec541bd00c4722a372edeae5d93342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.434414 16221 tablet_bootstrap.cc:492] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Bootstrap starting.
I20260812 21:08:34.434751 16222 tablet_bootstrap.cc:492] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Bootstrap starting.
I20260812 21:08:34.435230 16221 tablet_bootstrap.cc:654] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.435608 16221 log.cc:824] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:34.435706 16222 tablet_bootstrap.cc:654] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.436033 16222 log.cc:824] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:34.436461 16221 tablet_bootstrap.cc:492] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: No bootstrap required, opened a new log
I20260812 21:08:34.436509 16224 tablet_bootstrap.cc:492] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Bootstrap starting.
I20260812 21:08:34.436533 16221 ts_tablet_manager.cc:1397] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:34.436794 16222 tablet_bootstrap.cc:492] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: No bootstrap required, opened a new log
I20260812 21:08:34.436854 16222 ts_tablet_manager.cc:1397] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:34.437362 16224 tablet_bootstrap.cc:654] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.437721 16224 log.cc:824] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:34.438623 16224 tablet_bootstrap.cc:492] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: No bootstrap required, opened a new log
I20260812 21:08:34.438695 16224 ts_tablet_manager.cc:1397] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:08:34.438854 16222 raft_consensus.cc:348] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.438985 16222 raft_consensus.cc:374] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.439018 16222 raft_consensus.cc:729] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.439131 16222 consensus_queue.cc:260] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.439245 16221 raft_consensus.cc:348] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.439386 16222 ts_tablet_manager.cc:1428] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:08:34.439427 16221 raft_consensus.cc:374] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.439474 16221 raft_consensus.cc:729] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.439498 16175 heartbeater.cc:503] Master 127.11.207.190:41623 was elected leader, sending a full tablet report...
I20260812 21:08:34.439607 16221 consensus_queue.cc:260] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.439858 16221 ts_tablet_manager.cc:1428] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:08:34.439978 15710 heartbeater.cc:503] Master 127.11.207.190:41623 was elected leader, sending a full tablet report...
I20260812 21:08:34.441120 16224 raft_consensus.cc:348] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.441244 16224 raft_consensus.cc:374] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.441284 16224 raft_consensus.cc:729] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec95c187506d430fb17be4d6da345a80, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.441399 16224 consensus_queue.cc:260] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.441650 16224 ts_tablet_manager.cc:1428] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 21:08:34.441694 15954 heartbeater.cc:503] Master 127.11.207.190:41623 was elected leader, sending a full tablet report...
I20260812 21:08:34.480021 16234 raft_consensus.cc:482] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:34.480177 16234 raft_consensus.cc:504] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.480584 16234 leader_election.cc:288] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941), dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:34.483817 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.480905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.11.207.129:49021)
0812 21:08:34.481105 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.481115 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.481173 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.481298 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.481298 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.481300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.481371 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.481373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.481980 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.481981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.482961 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.482964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.483546 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.483547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.483619 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.483624 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.483683 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.483721 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":140,"threads_started":1}
I20260812 21:08:34.483815 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.481037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:49021 (local address 127.11.207.131:33413)
0812 21:08:34.481101 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.481103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.481174 (+ 71us) server_negotiation.cc:407] Connection header received
0812 21:08:34.481223 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.481224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.481233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.481290 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.481433 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.481908 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.483036 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.483469 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.483532 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.483540 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.483691 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.483734 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:08:34.484022 15510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.480790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44941 (local address 127.11.207.129:36529)
0812 21:08:34.480863 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.480874 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.480921 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.481152 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.481153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.481155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.481237 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.481240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.482113 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.482115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.483080 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.483083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.483715 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.483716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.483845 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.483849 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.483914 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.483953 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 21:08:34.484048 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.480921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:36529 (local address 127.11.207.130:44941)
0812 21:08:34.480988 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.480990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.481015 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:08:34.481047 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.481048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.481058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.481121 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.481302 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.482010 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.483165 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.483644 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.483706 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.483713 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.483941 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.483989 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:08:34.484383 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abec541bd00c4722a372edeae5d93342" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" is_pre_election: true
I20260812 21:08:34.484462 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abec541bd00c4722a372edeae5d93342" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80" is_pre_election: true
I20260812 21:08:34.484561 16099 raft_consensus.cc:2454] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 0.
I20260812 21:08:34.484601 15880 raft_consensus.cc:2454] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 0.
I20260812 21:08:34.484865 15505 leader_election.cc:302] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ec1042b25f40dc88d7e1ba85767191, dd67a13d15d048e18679ceb7e11c1ce9; no voters:
I20260812 21:08:34.485090 16234 raft_consensus.cc:2781] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:34.485153 16234 raft_consensus.cc:482] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:34.485177 16234 raft_consensus.cc:3037] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:34.486056 16234 raft_consensus.cc:504] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.486172 16234 leader_election.cc:288] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 election: Requested vote from peers ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941), dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:34.486373 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abec541bd00c4722a372edeae5d93342" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd67a13d15d048e18679ceb7e11c1ce9"
I20260812 21:08:34.486373 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abec541bd00c4722a372edeae5d93342" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80"
I20260812 21:08:34.486449 15880 raft_consensus.cc:3037] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:34.486450 16099 raft_consensus.cc:3037] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:34.487214 15880 raft_consensus.cc:2454] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 1.
I20260812 21:08:34.487389 15505 leader_election.cc:302] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ec1042b25f40dc88d7e1ba85767191, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:34.487413 16099 raft_consensus.cc:2454] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 1.
I20260812 21:08:34.487465 16234 raft_consensus.cc:2781] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:34.487627 16234 raft_consensus.cc:686] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 LEADER]: Becoming Leader. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Running, Role: LEADER
I20260812 21:08:34.487727 16234 consensus_queue.cc:237] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.488617 15346 catalog_manager.cc:5495] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129). New cstate: current_term: 1 leader_uuid: "58ec1042b25f40dc88d7e1ba85767191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
W20260812 21:08:34.490741 15956 tablet.cc:2369] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 21:08:34 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568913, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:34.531998 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.525297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41623 (local address 127.0.0.1:59318)
0812 21:08:34.525547 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.525557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.525632 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.525794 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.525795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.526122 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:34.526172 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.526192 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.526835 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.526836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.527357 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.527359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.527668 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.527669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.527722 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.528228 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:34.528233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:34.529658 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:34.531016 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.531024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.531027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.531212 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.531379 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:34.531379 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:34.531380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:34.531431 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:34.531632 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:34.531640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:34.531692 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.531746 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.531874 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":3,"thread_start_us":175,"threads_started":1}
I20260812 21:08:34.532007 15429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.525376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59318 (local address 127.11.207.190:41623)
0812 21:08:34.525444 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.525446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.525627 (+ 181us) server_negotiation.cc:407] Connection header received
0812 21:08:34.525719 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.525720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.525730 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.525778 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:34.526278 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.526764 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.527433 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.527611 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.527665 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.529750 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:34.529751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:34.529751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:34.529758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:34.530941 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.531276 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.531277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.531278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.531327 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:34.531478 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:34.531479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:34.531479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:34.531582 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:34.531636 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.531774 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.531932 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":43}
I20260812 21:08:34.534466 15346 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:59318:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 21:08:34.534698 15346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:34.536275 15844 tablet_service.cc:1467] Processing CreateTablet for tablet 471f10bc888b464881916337ad54348b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=687771428dfd4d7da53ca53f34101fa8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:08:34.536283 15580 tablet_service.cc:1467] Processing CreateTablet for tablet 471f10bc888b464881916337ad54348b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=687771428dfd4d7da53ca53f34101fa8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:08:34.536288 16062 tablet_service.cc:1467] Processing CreateTablet for tablet 471f10bc888b464881916337ad54348b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=687771428dfd4d7da53ca53f34101fa8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:08:34.536499 16062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471f10bc888b464881916337ad54348b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.536499 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471f10bc888b464881916337ad54348b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.536499 15844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471f10bc888b464881916337ad54348b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.537679 16222 tablet_bootstrap.cc:492] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: Bootstrap starting.
I20260812 21:08:34.538157 16224 tablet_bootstrap.cc:492] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: Bootstrap starting.
I20260812 21:08:34.538359 16222 tablet_bootstrap.cc:654] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.538900 16224 tablet_bootstrap.cc:654] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.539278 16222 tablet_bootstrap.cc:492] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: No bootstrap required, opened a new log
I20260812 21:08:34.539319 16222 ts_tablet_manager.cc:1397] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:34.539346 16221 tablet_bootstrap.cc:492] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: Bootstrap starting.
I20260812 21:08:34.539616 16224 tablet_bootstrap.cc:492] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: No bootstrap required, opened a new log
I20260812 21:08:34.539661 16224 ts_tablet_manager.cc:1397] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 21:08:34.539604 16222 raft_consensus.cc:348] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.539690 16222 raft_consensus.cc:374] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.539710 16222 raft_consensus.cc:729] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.539752 16222 consensus_queue.cc:260] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.539834 16222 ts_tablet_manager.cc:1428] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:34.539896 16224 raft_consensus.cc:348] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.539956 16224 raft_consensus.cc:374] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.539970 16224 raft_consensus.cc:729] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec95c187506d430fb17be4d6da345a80, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.540025 16224 consensus_queue.cc:260] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.540148 16224 ts_tablet_manager.cc:1428] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.540032 16232 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.540199 16221 tablet_bootstrap.cc:654] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.540195 16232 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:34.540213 16235 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.540308 16235 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:34.541247 16221 tablet_bootstrap.cc:492] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: No bootstrap required, opened a new log
I20260812 21:08:34.541291 16221 ts_tablet_manager.cc:1397] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:08:34.541457 16221 raft_consensus.cc:348] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.541528 16221 raft_consensus.cc:374] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.541540 16221 raft_consensus.cc:729] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.541574 16221 consensus_queue.cc:260] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.541671 16221 ts_tablet_manager.cc:1428] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.541702 16234 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.541771 16234 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 21:08:34.579178 15711 tablet.cc:2369] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:08:34.641072 16176 tablet.cc:2369] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:34.855778 16232 raft_consensus.cc:482] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:34.855899 16232 raft_consensus.cc:504] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.856276 16232 leader_election.cc:288] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237), ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941)
I20260812 21:08:34.859203 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.856606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:51211 (local address 127.11.207.130:44941)
0812 21:08:34.856661 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.856664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.856686 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:08:34.856786 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.856787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.856798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.856849 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.857019 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.857529 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.858494 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.858858 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.858933 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.858942 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.859086 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.859129 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:08:34.859302 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.856541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44941 (local address 127.11.207.131:51211)
0812 21:08:34.856607 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.856619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.856685 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.856870 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.856871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.856873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.856949 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.856952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.857622 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.857623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.858399 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.858401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.858936 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.858937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.859021 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.859025 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.859077 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.859120 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 21:08:34.859513 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.856541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:44237 (local address 127.11.207.131:35421)
0812 21:08:34.856812 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.856821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.856894 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.857026 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.857027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.857029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:34.857138 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.857140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.857761 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.857762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.858720 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.858722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.859250 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.859251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.859318 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.859322 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.859403 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.859438 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":20,"thread_start_us":110,"threads_started":1}
I20260812 21:08:34.859567 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.856676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:35421 (local address 127.11.207.129:44237)
0812 21:08:34.856719 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.856721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.856888 (+ 167us) server_negotiation.cc:407] Connection header received
0812 21:08:34.856957 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.856959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.856968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.857020 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:34.857212 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.857687 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.858802 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.859153 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.859219 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.859227 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.859444 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.859501 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:34.859570 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471f10bc888b464881916337ad54348b" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80" is_pre_election: true
I20260812 21:08:34.859680 15880 raft_consensus.cc:2454] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 0.
I20260812 21:08:34.859870 15619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471f10bc888b464881916337ad54348b" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ec1042b25f40dc88d7e1ba85767191" is_pre_election: true
I20260812 21:08:34.859917 15998 leader_election.cc:302] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd67a13d15d048e18679ceb7e11c1ce9, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:34.860008 15619 raft_consensus.cc:2454] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 0.
I20260812 21:08:34.860051 16232 raft_consensus.cc:2781] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:34.860105 16232 raft_consensus.cc:482] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:34.860126 16232 raft_consensus.cc:3037] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:34.860926 16232 raft_consensus.cc:504] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.861056 16232 leader_election.cc:288] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 election: Requested vote from peers 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237), ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941)
I20260812 21:08:34.861236 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471f10bc888b464881916337ad54348b" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80"
I20260812 21:08:34.861225 15619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471f10bc888b464881916337ad54348b" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ec1042b25f40dc88d7e1ba85767191"
I20260812 21:08:34.861310 15880 raft_consensus.cc:3037] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:34.861321 15619 raft_consensus.cc:3037] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:34.861980 15880 raft_consensus.cc:2454] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 1.
I20260812 21:08:34.862080 15619 raft_consensus.cc:2454] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 1.
I20260812 21:08:34.862157 15998 leader_election.cc:302] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd67a13d15d048e18679ceb7e11c1ce9, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:34.862269 16232 raft_consensus.cc:2781] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:34.862404 16232 raft_consensus.cc:686] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 LEADER]: Becoming Leader. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Running, Role: LEADER
I20260812 21:08:34.862509 16232 consensus_queue.cc:237] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.863085 16259 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: TxnStatusTablet state changed. Reason: New leader dd67a13d15d048e18679ceb7e11c1ce9. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.863183 16259 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:08:34.863430 16262 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 21:08:34.863546 16262 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 21:08:34.863504 15347 catalog_manager.cc:5495] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } health_report { overall_health: HEALTHY } } }
I20260812 21:08:34.893759 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.891008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.11.207.131:33413)
0812 21:08:34.891064 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.891065 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.891074 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:34.891167 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.891168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.891187 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.891241 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:34.891373 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.891858 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.892977 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.893168 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.893221 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.893439 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.893535 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.893594 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.893663 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.893685 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:08:34.893781 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.890920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.0.0.1:51956)
0812 21:08:34.891005 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.891019 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.891076 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.891256 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.891257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.891259 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:34.891316 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.891318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.891932 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.891933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.892897 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.892902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.893241 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.893242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.893316 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.893334 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.893589 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.893595 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.893642 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.893697 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 21:08:34.895726 15619 raft_consensus.cc:1261] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Refusing update from remote peer dd67a13d15d048e18679ceb7e11c1ce9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:34.895728 15880 raft_consensus.cc:1261] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Refusing update from remote peer dd67a13d15d048e18679ceb7e11c1ce9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:34.896102 16259 consensus_queue.cc:1035] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:34.896271 16232 consensus_queue.cc:1035] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:34.897549 16235 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: TxnStatusTablet state changed. Reason: New leader dd67a13d15d048e18679ceb7e11c1ce9. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.897644 16235 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:34.898314 16234 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: TxnStatusTablet state changed. Reason: New leader dd67a13d15d048e18679ceb7e11c1ce9. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.898414 16234 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:34.898978 16259 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.899076 16259 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:08:34.899133 16234 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.899235 16234 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P 58ec1042b25f40dc88d7e1ba85767191: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:34.899289 16232 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.899366 16232 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P dd67a13d15d048e18679ceb7e11c1ce9: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:08:34.899701 16269 mvcc.cc:205] Tried to move back new op lower bound from 7317786275410411520 to 7317786275277348864. Current Snapshot: MvccSnapshot[applied={T|T < 7317786275410411520 or (T in {7317786275410411520})}]
I20260812 21:08:34.900043 16235 tablet_replica.cc:439] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } } }
I20260812 21:08:34.900133 16235 tablet_replica.cc:442] T 471f10bc888b464881916337ad54348b P ec95c187506d430fb17be4d6da345a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:08:34.900751 16268 mvcc.cc:205] Tried to move back new op lower bound from 7317786275410411520 to 7317786275277348864. Current Snapshot: MvccSnapshot[applied={T|T < 7317786275410411520}]
I20260812 21:08:34.902712 16234 consensus_queue.cc:1035] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:34.904193 16238 consensus_queue.cc:1035] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:34.905469 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.901750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.11.207.129:44237)
0812 21:08:34.901789 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.901791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.901800 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:34.901833 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.901834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.901852 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.901927 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:34.902017 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.902784 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.904391 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.904704 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.904765 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.905028 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.905147 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.905228 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.905379 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.905406 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:34.910276 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.907912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:47297 (local address 127.11.207.131:33413)
0812 21:08:34.907961 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.907963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.908063 (+ 100us) server_negotiation.cc:407] Connection header received
0812 21:08:34.908110 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.908111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.908121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.908165 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:34.908280 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.908784 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.909606 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.909820 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.909878 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.910052 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.910086 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.910126 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.910188 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.910208 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:08:34.910387 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.907819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.11.207.129:47297)
0812 21:08:34.907963 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.907974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.908066 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.908166 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.908167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.908168 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:34.908227 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.908229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.908865 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.908866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.909511 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.909515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.909887 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.909888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.909958 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.909972 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.910145 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.910146 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.910186 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.910228 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260812 21:08:34.919870 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.917344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:36389 (local address 127.11.207.129:44237)
0812 21:08:34.917388 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:34.917391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:34.917535 (+ 144us) server_negotiation.cc:407] Connection header received
0812 21:08:34.917579 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:34.917580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:34.917590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:34.917640 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:34.917781 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.918275 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.919246 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.919445 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.919497 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.919646 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.919681 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.919726 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:34.919775 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 21:08:34.919795 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:34.919900 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:34.917293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:44237 (local address 127.11.207.131:36389)
0812 21:08:34.917476 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:34.917484 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:34.917535 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:34.917643 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:34.917644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:34.917645 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:34.917731 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.917732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.918343 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.918344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.919177 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:34.919180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:34.919500 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:34.919501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:34.919565 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:34.919581 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:34.919731 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:34.919733 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:34.919773 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 21:08:34.919817 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260812 21:08:34.929654 15346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:34.929788 15346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:34.930624 15346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Sending DeleteTablet for 3 replicas of tablet abec541bd00c4722a372edeae5d93342
I20260812 21:08:34.931174 15844 tablet_service.cc:1514] Processing DeleteTablet for tablet abec541bd00c4722a372edeae5d93342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:34 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:32806
I20260812 21:08:34.931408 15580 tablet_service.cc:1514] Processing DeleteTablet for tablet abec541bd00c4722a372edeae5d93342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:34 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260812 21:08:34.931602 16062 tablet_service.cc:1514] Processing DeleteTablet for tablet abec541bd00c4722a372edeae5d93342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:34 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51942
I20260812 21:08:34.931854 16298 tablet_replica.cc:330] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: stopping tablet replica
I20260812 21:08:34.931825 15346 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:34.931968 15346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:34.932168 16298 raft_consensus.cc:2229] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:34.932310 16298 raft_consensus.cc:2258] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:34.933209 16296 tablet_replica.cc:330] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: stopping tablet replica
I20260812 21:08:34.933252 15580 tablet_service.cc:1467] Processing CreateTablet for tablet 00cf5f552873470c9039e145bbf3fea4 (DEFAULT_TABLE table=test-table [id=70af516af0b541959f4398f66a6f7d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:34.933338 16296 raft_consensus.cc:2229] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:34.933425 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00cf5f552873470c9039e145bbf3fea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.933465 16296 raft_consensus.cc:2258] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:34.935338 16221 tablet_bootstrap.cc:492] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Bootstrap starting.
I20260812 21:08:34.935672 16296 ts_tablet_manager.cc:1905] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:34.935928 16221 tablet_bootstrap.cc:654] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.936957 15844 tablet_service.cc:1467] Processing CreateTablet for tablet 00cf5f552873470c9039e145bbf3fea4 (DEFAULT_TABLE table=test-table [id=70af516af0b541959f4398f66a6f7d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:34.937074 15844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00cf5f552873470c9039e145bbf3fea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.937959 16296 ts_tablet_manager.cc:1918] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:34.938033 16296 log.cc:1196] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/abec541bd00c4722a372edeae5d93342
I20260812 21:08:34.938297 16296 ts_tablet_manager.cc:1939] T abec541bd00c4722a372edeae5d93342 P ec95c187506d430fb17be4d6da345a80: Deleting consensus metadata
I20260812 21:08:34.938450 16224 tablet_bootstrap.cc:492] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Bootstrap starting.
I20260812 21:08:34.938503 16062 tablet_service.cc:1467] Processing CreateTablet for tablet 00cf5f552873470c9039e145bbf3fea4 (DEFAULT_TABLE table=test-table [id=70af516af0b541959f4398f66a6f7d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:34.938619 16062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00cf5f552873470c9039e145bbf3fea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:34.938963 16224 tablet_bootstrap.cc:654] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.939986 16222 tablet_bootstrap.cc:492] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Bootstrap starting.
I20260812 21:08:34.940219 16224 tablet_bootstrap.cc:492] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: No bootstrap required, opened a new log
I20260812 21:08:34.940284 16224 ts_tablet_manager.cc:1397] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 21:08:34.940446 16222 tablet_bootstrap.cc:654] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:34.940459 15327 catalog_manager.cc:4841] TS ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941): tablet abec541bd00c4722a372edeae5d93342 (table test-table [id=16146069478f4015963e255a6c2c6955]) successfully deleted
I20260812 21:08:34.940532 16224 raft_consensus.cc:348] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.940618 16224 raft_consensus.cc:374] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.940642 16224 raft_consensus.cc:729] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec95c187506d430fb17be4d6da345a80, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.940682 16224 consensus_queue.cc:260] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.940768 16224 ts_tablet_manager.cc:1428] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:34.941108 16298 ts_tablet_manager.cc:1905] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:34.942595 16298 ts_tablet_manager.cc:1918] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:34.942731 16298 log.cc:1196] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/abec541bd00c4722a372edeae5d93342
I20260812 21:08:34.943522 16298 ts_tablet_manager.cc:1939] T abec541bd00c4722a372edeae5d93342 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting consensus metadata
I20260812 21:08:34.943966 16221 tablet_bootstrap.cc:492] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: No bootstrap required, opened a new log
I20260812 21:08:34.944020 16221 ts_tablet_manager.cc:1397] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260812 21:08:34.944193 16221 raft_consensus.cc:348] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.944262 16221 raft_consensus.cc:374] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.944280 16221 raft_consensus.cc:729] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.944320 16221 consensus_queue.cc:260] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.944427 16221 ts_tablet_manager.cc:1428] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:34.944474 16297 tablet_replica.cc:330] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: stopping tablet replica
I20260812 21:08:34.944598 16297 raft_consensus.cc:2229] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:34.944731 16297 raft_consensus.cc:2258] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:34.945078 16297 ts_tablet_manager.cc:1905] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:34.944726 16222 tablet_bootstrap.cc:492] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: No bootstrap required, opened a new log
I20260812 21:08:34.945226 16222 ts_tablet_manager.cc:1397] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 21:08:34.945647 16222 raft_consensus.cc:348] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.945739 16222 raft_consensus.cc:374] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:34.945760 16222 raft_consensus.cc:729] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Initialized, Role: FOLLOWER
I20260812 21:08:34.945801 16222 consensus_queue.cc:260] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:34.945916 16222 ts_tablet_manager.cc:1428] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:34.946597 15327 catalog_manager.cc:4841] TS dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413): tablet abec541bd00c4722a372edeae5d93342 (table test-table [id=16146069478f4015963e255a6c2c6955]) successfully deleted
I20260812 21:08:34.946842 16297 ts_tablet_manager.cc:1918] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:34.946929 16297 log.cc:1196] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/abec541bd00c4722a372edeae5d93342
I20260812 21:08:34.947150 16297 ts_tablet_manager.cc:1939] T abec541bd00c4722a372edeae5d93342 P 58ec1042b25f40dc88d7e1ba85767191: Deleting consensus metadata
I20260812 21:08:34.947541 15319 catalog_manager.cc:4841] TS 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237): tablet abec541bd00c4722a372edeae5d93342 (table test-table [id=16146069478f4015963e255a6c2c6955]) successfully deleted
I20260812 21:08:35.014487 16259 raft_consensus.cc:482] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:35.014631 16259 raft_consensus.cc:504] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:35.014791 16259 leader_election.cc:288] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237), ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941)
I20260812 21:08:35.015062 15619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00cf5f552873470c9039e145bbf3fea4" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ec1042b25f40dc88d7e1ba85767191" is_pre_election: true
I20260812 21:08:35.015074 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00cf5f552873470c9039e145bbf3fea4" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80" is_pre_election: true
I20260812 21:08:35.015159 15619 raft_consensus.cc:2454] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 0.
I20260812 21:08:35.015188 15880 raft_consensus.cc:2454] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 0.
I20260812 21:08:35.015316 15995 leader_election.cc:302] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ec1042b25f40dc88d7e1ba85767191, dd67a13d15d048e18679ceb7e11c1ce9; no voters:
I20260812 21:08:35.015427 16259 raft_consensus.cc:2781] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:35.015458 16259 raft_consensus.cc:482] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:35.015473 16259 raft_consensus.cc:3037] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:35.016331 16259 raft_consensus.cc:504] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:35.016515 16259 leader_election.cc:288] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 election: Requested vote from peers 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237), ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941)
I20260812 21:08:35.016692 15619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00cf5f552873470c9039e145bbf3fea4" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ec1042b25f40dc88d7e1ba85767191"
I20260812 21:08:35.016767 15619 raft_consensus.cc:3037] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:35.016722 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00cf5f552873470c9039e145bbf3fea4" candidate_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80"
I20260812 21:08:35.016812 15880 raft_consensus.cc:3037] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:35.017639 15880 raft_consensus.cc:2454] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 1.
I20260812 21:08:35.017643 15619 raft_consensus.cc:2454] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd67a13d15d048e18679ceb7e11c1ce9 in term 1.
I20260812 21:08:35.017822 15998 leader_election.cc:302] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd67a13d15d048e18679ceb7e11c1ce9, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:35.018080 16259 raft_consensus.cc:2781] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:35.018146 16259 raft_consensus.cc:686] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 LEADER]: Becoming Leader. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Running, Role: LEADER
I20260812 21:08:35.018204 16259 consensus_queue.cc:237] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:35.018808 15346 catalog_manager.cc:5495] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } health_report { overall_health: HEALTHY } } }
I20260812 21:08:35.032632 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:35.030022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.11.207.131:33413)
0812 21:08:35.030090 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:35.030092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:35.030102 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:35.030124 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:35.030125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:35.030142 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:35.030210 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:35.030338 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:35.031058 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:35.031789 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:35.032080 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:35.032153 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:35.032328 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:35.032391 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:35.032456 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:35.032519 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 21:08:35.032544 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:08:35.037432 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:35.035190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:34471 (local address 127.11.207.131:33413)
0812 21:08:35.035234 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:35.035236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:35.035245 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:35.035281 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:35.035281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:35.035289 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:35.035327 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:35.035430 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:35.035850 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:35.036840 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:35.037015 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:35.037065 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:35.037222 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:35.037245 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:35.037276 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:35.037337 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:08:35.037357 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260812 21:08:35.037511 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:35.035106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.11.207.131:34471)
0812 21:08:35.035176 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:35.035187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:35.035227 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:35.035332 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:35.035332 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:35.035334 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:35.035375 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:35.035377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:35.035916 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:35.035917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:35.036765 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:35.036768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:35.037083 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:35.037084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:35.037137 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:35.037145 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:35.037280 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:35.037282 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:35.037335 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:08:35.037389 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":5}
I20260812 21:08:35.039491 15880 raft_consensus.cc:1261] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Refusing update from remote peer dd67a13d15d048e18679ceb7e11c1ce9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:35.039510 15619 raft_consensus.cc:1261] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Refusing update from remote peer dd67a13d15d048e18679ceb7e11c1ce9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:35.039727 16232 consensus_queue.cc:1035] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:35.039839 16306 consensus_queue.cc:1035] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:35.041069 16265 mvcc.cc:205] Tried to move back new op lower bound from 7317786276000014336 to 7317786275915042816. Current Snapshot: MvccSnapshot[applied={T|T < 7317786276000014336}]
W20260812 21:08:35.044920 16265 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 21:08:36.565034 15347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:36.565135 15347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:36.565992 15347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Sending DeleteTablet for 3 replicas of tablet 00cf5f552873470c9039e145bbf3fea4
I20260812 21:08:36.566227 15580 tablet_service.cc:1514] Processing DeleteTablet for tablet 00cf5f552873470c9039e145bbf3fea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:36 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260812 21:08:36.566330 15844 tablet_service.cc:1514] Processing DeleteTablet for tablet 00cf5f552873470c9039e145bbf3fea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:36 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:32806
I20260812 21:08:36.566323 16061 tablet_service.cc:1514] Processing DeleteTablet for tablet 00cf5f552873470c9039e145bbf3fea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:36 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51942
I20260812 21:08:36.566562 16349 tablet_replica.cc:330] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: stopping tablet replica
I20260812 21:08:36.566574 16348 tablet_replica.cc:330] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: stopping tablet replica
I20260812 21:08:36.566620 16349 raft_consensus.cc:2229] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:36.566663 16349 raft_consensus.cc:2258] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:36.566684 16348 raft_consensus.cc:2229] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:36.566865 16348 raft_consensus.cc:2258] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:36.566900 16349 ts_tablet_manager.cc:1905] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:36.567123 16348 ts_tablet_manager.cc:1905] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:36.568547 16349 ts_tablet_manager.cc:1918] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:08:36.568612 16349 log.cc:1196] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/00cf5f552873470c9039e145bbf3fea4
I20260812 21:08:36.568779 16348 ts_tablet_manager.cc:1918] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:08:36.568849 16348 log.cc:1196] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/00cf5f552873470c9039e145bbf3fea4
I20260812 21:08:36.568837 16349 ts_tablet_manager.cc:1939] T 00cf5f552873470c9039e145bbf3fea4 P ec95c187506d430fb17be4d6da345a80: Deleting consensus metadata
I20260812 21:08:36.569185 15327 catalog_manager.cc:4841] TS ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941): tablet 00cf5f552873470c9039e145bbf3fea4 (table test-table [id=70af516af0b541959f4398f66a6f7d3e]) successfully deleted
I20260812 21:08:36.569202 16348 ts_tablet_manager.cc:1939] T 00cf5f552873470c9039e145bbf3fea4 P 58ec1042b25f40dc88d7e1ba85767191: Deleting consensus metadata
I20260812 21:08:36.569521 16351 tablet_replica.cc:330] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: stopping tablet replica
I20260812 21:08:36.569566 15319 catalog_manager.cc:4841] TS 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237): tablet 00cf5f552873470c9039e145bbf3fea4 (table test-table [id=70af516af0b541959f4398f66a6f7d3e]) successfully deleted
I20260812 21:08:36.569593 16351 raft_consensus.cc:2229] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:36.569712 16351 raft_consensus.cc:2258] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:36.569967 16351 ts_tablet_manager.cc:1905] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:36.571199 16351 ts_tablet_manager.cc:1918] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:08:36.571254 16351 log.cc:1196] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/00cf5f552873470c9039e145bbf3fea4
I20260812 21:08:36.571463 16351 ts_tablet_manager.cc:1939] T 00cf5f552873470c9039e145bbf3fea4 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting consensus metadata
I20260812 21:08:36.571782 15327 catalog_manager.cc:4841] TS dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413): tablet 00cf5f552873470c9039e145bbf3fea4 (table test-table [id=70af516af0b541959f4398f66a6f7d3e]) successfully deleted
Aug 12 21:08:36 dist-test-slave-lkx2 krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568916, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:08:36 dist-test-slave-lkx2 krb5kdc[15266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568916, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:36.606925 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.586034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.11.207.190:41623)
0812 21:08:36.586204 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.586206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.586217 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:36.586285 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.586286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.586298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.586365 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:36.586878 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.587400 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.588114 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.588332 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.588421 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.604493 (+ 16072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:36.604496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:36.604496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:36.604505 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:36.605859 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:36.606197 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:36.606197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:36.606198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:36.606253 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:36.606425 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:36.606426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:36.606427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:36.606573 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:36.606666 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.606752 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.606835 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260812 21:08:36.608529 15347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:36.608742 15347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:36.610154 15580 tablet_service.cc:1467] Processing CreateTablet for tablet b0bb5ef42d454b3e8f030caf06b888b0 (DEFAULT_TABLE table=test-table [id=e7bb8e5f5ef44a40943f095a84fc6e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:36.610154 15844 tablet_service.cc:1467] Processing CreateTablet for tablet b0bb5ef42d454b3e8f030caf06b888b0 (DEFAULT_TABLE table=test-table [id=e7bb8e5f5ef44a40943f095a84fc6e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:36.610185 16061 tablet_service.cc:1467] Processing CreateTablet for tablet b0bb5ef42d454b3e8f030caf06b888b0 (DEFAULT_TABLE table=test-table [id=e7bb8e5f5ef44a40943f095a84fc6e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:36.610303 15844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0bb5ef42d454b3e8f030caf06b888b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:36.610304 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0bb5ef42d454b3e8f030caf06b888b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:36.610317 16061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0bb5ef42d454b3e8f030caf06b888b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:36.611581 16369 tablet_bootstrap.cc:492] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Bootstrap starting.
I20260812 21:08:36.611794 16374 tablet_bootstrap.cc:492] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Bootstrap starting.
I20260812 21:08:36.611804 16373 tablet_bootstrap.cc:492] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Bootstrap starting.
I20260812 21:08:36.612089 16369 tablet_bootstrap.cc:654] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:36.612161 16374 tablet_bootstrap.cc:654] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:36.612284 16373 tablet_bootstrap.cc:654] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:36.612799 16374 tablet_bootstrap.cc:492] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: No bootstrap required, opened a new log
I20260812 21:08:36.612849 16374 ts_tablet_manager.cc:1397] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:36.612861 16373 tablet_bootstrap.cc:492] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: No bootstrap required, opened a new log
I20260812 21:08:36.612905 16373 ts_tablet_manager.cc:1397] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:36.612967 16374 raft_consensus.cc:348] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.613034 16374 raft_consensus.cc:374] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:36.613051 16374 raft_consensus.cc:729] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Initialized, Role: FOLLOWER
I20260812 21:08:36.613044 16373 raft_consensus.cc:348] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.613109 16373 raft_consensus.cc:374] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:36.613101 16374 consensus_queue.cc:260] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.613127 16373 raft_consensus.cc:729] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec95c187506d430fb17be4d6da345a80, State: Initialized, Role: FOLLOWER
I20260812 21:08:36.613183 16373 consensus_queue.cc:260] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.613281 16374 ts_tablet_manager.cc:1428] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:36.613363 16373 ts_tablet_manager.cc:1428] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:36.613519 16369 tablet_bootstrap.cc:492] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: No bootstrap required, opened a new log
I20260812 21:08:36.613577 16369 ts_tablet_manager.cc:1397] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:36.613706 16369 raft_consensus.cc:348] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.613780 16369 raft_consensus.cc:374] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:36.613807 16369 raft_consensus.cc:729] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Initialized, Role: FOLLOWER
I20260812 21:08:36.613849 16369 consensus_queue.cc:260] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.613945 16369 ts_tablet_manager.cc:1428] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:36.673580 16382 raft_consensus.cc:482] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:36.673691 16382 raft_consensus.cc:504] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.673878 16382 leader_election.cc:288] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941), dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:36.674121 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0bb5ef42d454b3e8f030caf06b888b0" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" is_pre_election: true
I20260812 21:08:36.674109 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0bb5ef42d454b3e8f030caf06b888b0" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80" is_pre_election: true
I20260812 21:08:36.674216 15880 raft_consensus.cc:2454] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 0.
I20260812 21:08:36.674216 16099 raft_consensus.cc:2454] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 0.
I20260812 21:08:36.674397 15505 leader_election.cc:302] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ec1042b25f40dc88d7e1ba85767191, dd67a13d15d048e18679ceb7e11c1ce9; no voters:
I20260812 21:08:36.674499 16382 raft_consensus.cc:2781] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:36.674543 16382 raft_consensus.cc:482] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:36.674567 16382 raft_consensus.cc:3037] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:36.675323 16382 raft_consensus.cc:504] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.675437 16382 leader_election.cc:288] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 election: Requested vote from peers ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941), dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:36.675624 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0bb5ef42d454b3e8f030caf06b888b0" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd67a13d15d048e18679ceb7e11c1ce9"
I20260812 21:08:36.675627 15880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0bb5ef42d454b3e8f030caf06b888b0" candidate_uuid: "58ec1042b25f40dc88d7e1ba85767191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec95c187506d430fb17be4d6da345a80"
I20260812 21:08:36.675747 15880 raft_consensus.cc:3037] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:36.675747 16099 raft_consensus.cc:3037] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:36.676568 15880 raft_consensus.cc:2454] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 1.
I20260812 21:08:36.676625 16099 raft_consensus.cc:2454] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ec1042b25f40dc88d7e1ba85767191 in term 1.
I20260812 21:08:36.676765 15505 leader_election.cc:302] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ec1042b25f40dc88d7e1ba85767191, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:36.676883 16382 raft_consensus.cc:2781] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:36.677057 16382 raft_consensus.cc:686] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 LEADER]: Becoming Leader. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Running, Role: LEADER
I20260812 21:08:36.677106 16382 consensus_queue.cc:237] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.677706 15346 catalog_manager.cc:5495] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129). New cstate: current_term: 1 leader_uuid: "58ec1042b25f40dc88d7e1ba85767191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
I20260812 21:08:36.690277 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.687963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.11.207.129:44237)
0812 21:08:36.688114 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.688116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.688124 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:36.688144 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.688144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.688164 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.688218 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:36.688386 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.688892 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.689646 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.689850 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.689895 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.689992 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:36.690019 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:36.690051 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.690188 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.690206 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260812 21:08:36.692530 16099 raft_consensus.cc:1261] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Refusing update from remote peer 58ec1042b25f40dc88d7e1ba85767191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:36.692540 15880 raft_consensus.cc:1261] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Refusing update from remote peer 58ec1042b25f40dc88d7e1ba85767191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:36.692721 16382 consensus_queue.cc:1035] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:36.692838 16386 consensus_queue.cc:1035] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:36.702587 15345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:36.702690 15345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:36.703249 15345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Sending DeleteTablet for 3 replicas of tablet b0bb5ef42d454b3e8f030caf06b888b0
I20260812 21:08:36.703469 15844 tablet_service.cc:1514] Processing DeleteTablet for tablet b0bb5ef42d454b3e8f030caf06b888b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:36 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:32806
I20260812 21:08:36.703469 15580 tablet_service.cc:1514] Processing DeleteTablet for tablet b0bb5ef42d454b3e8f030caf06b888b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:36 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260812 21:08:36.703585 16061 tablet_service.cc:1514] Processing DeleteTablet for tablet b0bb5ef42d454b3e8f030caf06b888b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:36 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51942
I20260812 21:08:36.703584 16349 tablet_replica.cc:330] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: stopping tablet replica
I20260812 21:08:36.703658 16349 raft_consensus.cc:2229] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:36.703689 16351 tablet_replica.cc:330] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: stopping tablet replica
I20260812 21:08:36.703704 16349 raft_consensus.cc:2258] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:36.703743 16351 raft_consensus.cc:2229] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:36.703784 16351 raft_consensus.cc:2258] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:36.703775 16348 tablet_replica.cc:330] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: stopping tablet replica
I20260812 21:08:36.703835 16348 raft_consensus.cc:2229] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:36.703907 16348 raft_consensus.cc:2258] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:36.704053 16349 ts_tablet_manager.cc:1905] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:36.704104 16351 ts_tablet_manager.cc:1905] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:36.704187 16348 ts_tablet_manager.cc:1905] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:36.705669 15345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:36.705871 15345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:36.705936 16348 ts_tablet_manager.cc:1918] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:36.705994 16348 log.cc:1196] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/b0bb5ef42d454b3e8f030caf06b888b0
I20260812 21:08:36.706097 16351 ts_tablet_manager.cc:1918] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:36.706180 16351 log.cc:1196] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/b0bb5ef42d454b3e8f030caf06b888b0
I20260812 21:08:36.706203 16348 ts_tablet_manager.cc:1939] T b0bb5ef42d454b3e8f030caf06b888b0 P 58ec1042b25f40dc88d7e1ba85767191: Deleting consensus metadata
I20260812 21:08:36.706398 16351 ts_tablet_manager.cc:1939] T b0bb5ef42d454b3e8f030caf06b888b0 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting consensus metadata
I20260812 21:08:36.706482 15319 catalog_manager.cc:4841] TS 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237): tablet b0bb5ef42d454b3e8f030caf06b888b0 (table test-table [id=e7bb8e5f5ef44a40943f095a84fc6e8d]) successfully deleted
I20260812 21:08:36.706686 15327 catalog_manager.cc:4841] TS dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413): tablet b0bb5ef42d454b3e8f030caf06b888b0 (table test-table [id=e7bb8e5f5ef44a40943f095a84fc6e8d]) successfully deleted
I20260812 21:08:36.706758 16349 ts_tablet_manager.cc:1918] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:08:36.706816 16349 log.cc:1196] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/b0bb5ef42d454b3e8f030caf06b888b0
I20260812 21:08:36.707022 16349 ts_tablet_manager.cc:1939] T b0bb5ef42d454b3e8f030caf06b888b0 P ec95c187506d430fb17be4d6da345a80: Deleting consensus metadata
I20260812 21:08:36.707077 16061 tablet_service.cc:1467] Processing CreateTablet for tablet b132a5d5769f4a84a28c3180c9aaec37 (DEFAULT_TABLE table=test-table [id=d289d84d709c4a48ad0c0c9b63e52ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:36.707075 15844 tablet_service.cc:1467] Processing CreateTablet for tablet b132a5d5769f4a84a28c3180c9aaec37 (DEFAULT_TABLE table=test-table [id=d289d84d709c4a48ad0c0c9b63e52ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:36.707170 15844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b132a5d5769f4a84a28c3180c9aaec37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:36.707180 16061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b132a5d5769f4a84a28c3180c9aaec37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:36.707284 15580 tablet_service.cc:1467] Processing CreateTablet for tablet b132a5d5769f4a84a28c3180c9aaec37 (DEFAULT_TABLE table=test-table [id=d289d84d709c4a48ad0c0c9b63e52ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:36.707325 15327 catalog_manager.cc:4841] TS ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941): tablet b0bb5ef42d454b3e8f030caf06b888b0 (table test-table [id=e7bb8e5f5ef44a40943f095a84fc6e8d]) successfully deleted
I20260812 21:08:36.707369 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b132a5d5769f4a84a28c3180c9aaec37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:36.708020 16373 tablet_bootstrap.cc:492] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Bootstrap starting.
I20260812 21:08:36.708437 16369 tablet_bootstrap.cc:492] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Bootstrap starting.
I20260812 21:08:36.708488 16374 tablet_bootstrap.cc:492] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191: Bootstrap starting.
I20260812 21:08:36.708772 16373 tablet_bootstrap.cc:654] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:36.708776 16369 tablet_bootstrap.cc:654] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:36.708776 16374 tablet_bootstrap.cc:654] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:36.709555 16369 tablet_bootstrap.cc:492] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: No bootstrap required, opened a new log
I20260812 21:08:36.709565 16373 tablet_bootstrap.cc:492] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: No bootstrap required, opened a new log
I20260812 21:08:36.709605 16369 ts_tablet_manager.cc:1397] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:36.709607 16373 ts_tablet_manager.cc:1397] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:08:36.709735 16374 tablet_bootstrap.cc:492] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191: No bootstrap required, opened a new log
I20260812 21:08:36.709725 16369 raft_consensus.cc:348] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.709790 16369 raft_consensus.cc:374] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:36.709785 16374 ts_tablet_manager.cc:1397] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:36.709766 16373 raft_consensus.cc:348] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.709815 16369 raft_consensus.cc:729] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd67a13d15d048e18679ceb7e11c1ce9, State: Initialized, Role: FOLLOWER
I20260812 21:08:36.709831 16373 raft_consensus.cc:374] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:36.709851 16373 raft_consensus.cc:729] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec95c187506d430fb17be4d6da345a80, State: Initialized, Role: FOLLOWER
I20260812 21:08:36.709858 16369 consensus_queue.cc:260] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.709890 16373 consensus_queue.cc:260] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.709944 16369 ts_tablet_manager.cc:1428] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:36.709916 16374 raft_consensus.cc:348] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.709976 16374 raft_consensus.cc:374] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:36.709972 16373 ts_tablet_manager.cc:1428] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:36.710002 16374 raft_consensus.cc:729] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ec1042b25f40dc88d7e1ba85767191, State: Initialized, Role: FOLLOWER
I20260812 21:08:36.710039 16374 consensus_queue.cc:260] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.710152 16374 ts_tablet_manager.cc:1428] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:36.784816 16384 raft_consensus.cc:482] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:36.784914 16384 raft_consensus.cc:504] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.785319 16384 leader_election.cc:288] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237), dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:36.788204 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.785542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:60493 (local address 127.11.207.129:44237)
0812 21:08:36.785595 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.785597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.785650 (+ 53us) server_negotiation.cc:407] Connection header received
0812 21:08:36.785774 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.785775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.785785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.785833 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:36.785988 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.786522 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.787506 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.787887 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.787949 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.787959 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.788095 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.788135 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:08:36.788215 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.785469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:44237 (local address 127.11.207.130:60493)
0812 21:08:36.785595 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:36.785605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:36.785701 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:36.785843 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:36.785844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:36.785845 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:36.785930 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.785932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.786593 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.786594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.787416 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.787418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.787959 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.787960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.788031 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.788035 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:36.788080 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:08:36.788119 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":47,"thread_start_us":67,"threads_started":1}
I20260812 21:08:36.788295 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.785650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:46979 (local address 127.11.207.131:33413)
0812 21:08:36.785778 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.785780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.785787 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:36.785812 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.785812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.785820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.785891 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:36.786039 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.786568 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.787597 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.787976 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.788038 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.788046 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.788179 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.788222 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260812 21:08:36.788509 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.785476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.11.207.130:46979)
0812 21:08:36.785633 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:36.785645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:36.785701 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:36.785904 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:36.785905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:36.785907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:36.785979 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.785981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.786674 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.786674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.787509 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.787511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.788047 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.788048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.788118 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.788121 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:08:36.788165 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:08:36.788202 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260812 21:08:36.788745 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b132a5d5769f4a84a28c3180c9aaec37" candidate_uuid: "ec95c187506d430fb17be4d6da345a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" is_pre_election: true
I20260812 21:08:36.788834 16099 raft_consensus.cc:2454] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec95c187506d430fb17be4d6da345a80 in term 0.
I20260812 21:08:36.788682 15619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b132a5d5769f4a84a28c3180c9aaec37" candidate_uuid: "ec95c187506d430fb17be4d6da345a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ec1042b25f40dc88d7e1ba85767191" is_pre_election: true
I20260812 21:08:36.788924 15619 raft_consensus.cc:2454] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec95c187506d430fb17be4d6da345a80 in term 0.
I20260812 21:08:36.789024 15761 leader_election.cc:302] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd67a13d15d048e18679ceb7e11c1ce9, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:36.789170 16384 raft_consensus.cc:2781] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:36.789213 16384 raft_consensus.cc:482] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:36.789232 16384 raft_consensus.cc:3037] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:36.789985 16384 raft_consensus.cc:504] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.790117 16384 leader_election.cc:288] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [CANDIDATE]: Term 1 election: Requested vote from peers 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237), dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413)
I20260812 21:08:36.790269 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b132a5d5769f4a84a28c3180c9aaec37" candidate_uuid: "ec95c187506d430fb17be4d6da345a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd67a13d15d048e18679ceb7e11c1ce9"
I20260812 21:08:36.790345 16099 raft_consensus.cc:3037] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:36.790313 15619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b132a5d5769f4a84a28c3180c9aaec37" candidate_uuid: "ec95c187506d430fb17be4d6da345a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ec1042b25f40dc88d7e1ba85767191"
I20260812 21:08:36.790396 15619 raft_consensus.cc:3037] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:36.790963 15619 raft_consensus.cc:2454] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec95c187506d430fb17be4d6da345a80 in term 1.
I20260812 21:08:36.791036 16099 raft_consensus.cc:2454] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec95c187506d430fb17be4d6da345a80 in term 1.
I20260812 21:08:36.791133 15755 leader_election.cc:302] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ec1042b25f40dc88d7e1ba85767191, ec95c187506d430fb17be4d6da345a80; no voters:
I20260812 21:08:36.791230 16384 raft_consensus.cc:2781] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:36.791392 16384 raft_consensus.cc:686] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 1 LEADER]: Becoming Leader. State: Replica: ec95c187506d430fb17be4d6da345a80, State: Running, Role: LEADER
I20260812 21:08:36.791460 16384 consensus_queue.cc:237] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } }
I20260812 21:08:36.792133 15346 catalog_manager.cc:5495] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec95c187506d430fb17be4d6da345a80 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "ec95c187506d430fb17be4d6da345a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec95c187506d430fb17be4d6da345a80" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 } health_report { overall_health: UNKNOWN } } }
I20260812 21:08:36.804774 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.802028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.11.207.130:44941)
0812 21:08:36.802200 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.802202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.802209 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:36.802245 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.802245 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.802264 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.802335 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:36.802427 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.803116 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.804014 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.804250 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.804315 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.804468 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:36.804586 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:36.804644 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.804661 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.804703 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260812 21:08:36.809437 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.807044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33413 (local address 127.11.207.130:51211)
0812 21:08:36.807210 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:36.807221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:36.807279 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:36.807394 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:36.807395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:36.807397 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:36.807465 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.807467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.808072 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.808073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.808673 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.808675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.809029 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.809030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.809092 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.809107 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:36.809263 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:36.809265 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:36.809301 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 21:08:36.809346 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260812 21:08:36.809422 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.807194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:51211 (local address 127.11.207.131:33413)
0812 21:08:36.807263 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.807265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.807274 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:36.807330 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.807331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.807342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.807388 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:36.807524 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.807990 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.808756 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.808948 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.809003 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.809181 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:36.809212 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:36.809252 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.809313 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.809339 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:08:36.811702 16099 raft_consensus.cc:1261] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Refusing update from remote peer ec95c187506d430fb17be4d6da345a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:36.811726 15619 raft_consensus.cc:1261] T b132a5d5769f4a84a28c3180c9aaec37 P 58ec1042b25f40dc88d7e1ba85767191 [term 1 FOLLOWER]: Refusing update from remote peer ec95c187506d430fb17be4d6da345a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:36.811980 16408 consensus_queue.cc:1035] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd67a13d15d048e18679ceb7e11c1ce9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:36.812114 16384 consensus_queue.cc:1035] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58ec1042b25f40dc88d7e1ba85767191" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:36.819099 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.816885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:41409 (local address 127.11.207.130:44941)
0812 21:08:36.816926 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:36.816929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:36.817032 (+ 103us) server_negotiation.cc:407] Connection header received
0812 21:08:36.817112 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:36.817113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:36.817125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:36.817171 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:36.817299 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.817763 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.818438 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.818681 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.818745 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.818869 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:36.818902 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:36.818947 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:36.819002 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 21:08:36.819027 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:36.819110 16430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:36.816832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44941 (local address 127.11.207.131:41409)
0812 21:08:36.816959 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:36.816970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:36.817059 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:36.817178 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:36.817179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:36.817180 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:08:36.817231 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.817233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.817823 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.817823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.818374 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:36.818375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:36.818742 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:36.818743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:36.818793 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:36.818797 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:08:36.818953 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:08:36.818955 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:36.818998 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:08:36.819048 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
W20260812 21:08:36.819399 16317 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 21:08:38.338069 15346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:38.338173 15346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:38.339039 15346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ed7e8c8f46c4f79b0ee27e7e386b4f7: Sending DeleteTablet for 3 replicas of tablet b132a5d5769f4a84a28c3180c9aaec37
I20260812 21:08:38.339289 15844 tablet_service.cc:1514] Processing DeleteTablet for tablet b132a5d5769f4a84a28c3180c9aaec37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:38 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:32806
I20260812 21:08:38.339303 15580 tablet_service.cc:1514] Processing DeleteTablet for tablet b132a5d5769f4a84a28c3180c9aaec37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:38 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260812 21:08:38.339416 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15473
I20260812 21:08:38.345229 16061 tablet_service.cc:1514] Processing DeleteTablet for tablet b132a5d5769f4a84a28c3180c9aaec37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:38 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51942
I20260812 21:08:38.345585 16469 tablet_replica.cc:330] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: stopping tablet replica
I20260812 21:08:38.345661 16469 raft_consensus.cc:2229] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:38.345777 16469 raft_consensus.cc:2258] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:38.346179 16469 ts_tablet_manager.cc:1905] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:38.348228 16469 ts_tablet_manager.cc:1918] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:08:38.348295 16469 log.cc:1196] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/b132a5d5769f4a84a28c3180c9aaec37
I20260812 21:08:38.348565 16469 ts_tablet_manager.cc:1939] T b132a5d5769f4a84a28c3180c9aaec37 P ec95c187506d430fb17be4d6da345a80: Deleting consensus metadata
I20260812 21:08:38.348955 15327 catalog_manager.cc:4841] TS ec95c187506d430fb17be4d6da345a80 (127.11.207.130:44941): tablet b132a5d5769f4a84a28c3180c9aaec37 (table test-table [id=d289d84d709c4a48ad0c0c9b63e52ed9]) successfully deleted
I20260812 21:08:38.352559 16473 tablet_replica.cc:330] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: stopping tablet replica
I20260812 21:08:38.352639 16473 raft_consensus.cc:2229] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:38.352682 16473 raft_consensus.cc:2258] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:38.352936 16473 ts_tablet_manager.cc:1905] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:38.354344 16473 ts_tablet_manager.cc:1918] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:08:38.354413 16473 log.cc:1196] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/b132a5d5769f4a84a28c3180c9aaec37
W20260812 21:08:38.354489 15319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:44237 (error 108)
I20260812 21:08:38.354621 16473 ts_tablet_manager.cc:1939] T b132a5d5769f4a84a28c3180c9aaec37 P dd67a13d15d048e18679ceb7e11c1ce9: Deleting consensus metadata
I20260812 21:08:38.354951 15327 catalog_manager.cc:4841] TS dd67a13d15d048e18679ceb7e11c1ce9 (127.11.207.131:33413): tablet b132a5d5769f4a84a28c3180c9aaec37 (table test-table [id=d289d84d709c4a48ad0c0c9b63e52ed9]) successfully deleted
I20260812 21:08:38.355180 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15728
I20260812 21:08:38.355204 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:38.354984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:44237 (local address 127.0.0.1:38766)
0812 21:08:38.355105 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:38.355158 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:44237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
W20260812 21:08:38.364990 15319 catalog_manager.cc:4584] TS 58ec1042b25f40dc88d7e1ba85767191 (127.11.207.129:44237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b132a5d5769f4a84a28c3180c9aaec37: Network error: Client connection negotiation failed: client connection to 127.11.207.129:44237: connect: Connection refused (error 111)
I20260812 21:08:38.372264 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15967
I20260812 21:08:38.387398 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15292
2026-08-12T21:08:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5989 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:38 dist-test-slave-lkx2 krb5kdc[16480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:38 dist-test-slave-lkx2 krb5kdc[16480](info): set up 2 sockets
Aug 12 21:08:38 dist-test-slave-lkx2 krb5kdc[16480](info): commencing operation
krb5kdc: starting...
W20260812 21:08:39.597653 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:39 dist-test-slave-lkx2 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:39.731379 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43967
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:39.815335 16501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:39.815543 16501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:39.815564 16501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:39.816774 16501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:39.816819 16501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:39.816833 16501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:39.816846 16501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:39.816870 16501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:39.818375 16501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:39.818619 16501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:39.818850 16501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:39.823645 16513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:39.823769 16501 server_base.cc:1034] running on GCE node
W20260812 21:08:39.823659 16509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:39.823652 16508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:39.824093 16501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:39.824383 16501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:39.825524 16501 hybrid_clock.cc:648] HybridClock initialized: now 1786568919825513 us; error 28 us; skew 500 ppm
Aug 12 21:08:39 dist-test-slave-lkx2 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568919, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:39.827980 16501 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:39.828444 16501 webserver.cc:460] Webserver started at http://127.11.207.190:38527/ using document root <none> and password file <none>
I20260812 21:08:39.828673 16501 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:39.828722 16501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:39.828847 16501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:39.830094 16501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "7135046166a14b7f8e37f6778d699aa4"
format_stamp: "Formatted at 2026-08-12 21:08:39 on dist-test-slave-lkx2"
server_key: "58a7eaec0909ad94bdbe046ef902223c"
server_key_iv: "affd8c64d4cc30372e79140d6147a018"
server_key_version: "encryptionkey@0"
I20260812 21:08:39.830466 16501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "7135046166a14b7f8e37f6778d699aa4"
format_stamp: "Formatted at 2026-08-12 21:08:39 on dist-test-slave-lkx2"
server_key: "58a7eaec0909ad94bdbe046ef902223c"
server_key_iv: "affd8c64d4cc30372e79140d6147a018"
server_key_version: "encryptionkey@0"
I20260812 21:08:39.831871 16501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:39.832674 16522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:39.832949 16501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:39.833047 16501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "7135046166a14b7f8e37f6778d699aa4"
format_stamp: "Formatted at 2026-08-12 21:08:39 on dist-test-slave-lkx2"
server_key: "58a7eaec0909ad94bdbe046ef902223c"
server_key_iv: "affd8c64d4cc30372e79140d6147a018"
server_key_version: "encryptionkey@0"
I20260812 21:08:39.833108 16501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:39.844446 16501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:39.846349 16501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:39.846494 16501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:39.850536 16501 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43967
I20260812 21:08:39.850570 16599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43967 every 8 connection(s)
I20260812 21:08:39.850929 16501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:39.851830 16602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:39.854794 16602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4: Bootstrap starting.
I20260812 21:08:39.855557 16602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:39.855887 16602 log.cc:824] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:39.856760 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16501
I20260812 21:08:39.856786 16602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4: No bootstrap required, opened a new log
I20260812 21:08:39.856846 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:39.857002 12094 external_mini_cluster.cc:1428] Setting key 728dc0c6232387be97942e44d3280816
I20260812 21:08:39.858632 16602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135046166a14b7f8e37f6778d699aa4" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43967 } }
I20260812 21:08:39.858772 16602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:39.858860 16602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7135046166a14b7f8e37f6778d699aa4, State: Initialized, Role: FOLLOWER
I20260812 21:08:39.858997 16602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135046166a14b7f8e37f6778d699aa4" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43967 } }
I20260812 21:08:39.859067 16602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:39.859100 16602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:39.859148 16602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:39.860000 16602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135046166a14b7f8e37f6778d699aa4" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43967 } }
I20260812 21:08:39.860102 16602 leader_election.cc:302] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7135046166a14b7f8e37f6778d699aa4; no voters:
I20260812 21:08:39.860275 16602 leader_election.cc:288] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:39.860318 16615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:39.860499 16615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [term 1 LEADER]: Becoming Leader. State: Replica: 7135046166a14b7f8e37f6778d699aa4, State: Running, Role: LEADER
I20260812 21:08:39.860584 16602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:39.860646 16615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135046166a14b7f8e37f6778d699aa4" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43967 } }
I20260812 21:08:39.861019 16615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7135046166a14b7f8e37f6778d699aa4. Latest consensus state: current_term: 1 leader_uuid: "7135046166a14b7f8e37f6778d699aa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135046166a14b7f8e37f6778d699aa4" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43967 } } }
I20260812 21:08:39.861259 16615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:39.861194 16620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7135046166a14b7f8e37f6778d699aa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135046166a14b7f8e37f6778d699aa4" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43967 } } }
I20260812 21:08:39.861488 16620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:39.861919 16629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:39.862502 16629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:39.864223 16629 catalog_manager.cc:1347] Generated new cluster ID: d458a3fa7d164bc788de4b4fa1b31681
I20260812 21:08:39.864279 16629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:39 dist-test-slave-lkx2 krb5kdc[16480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:39.869081 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:39.857520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.11.207.190:43967)
0812 21:08:39.857716 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:39.857724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:39.857750 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:08:39.858611 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:39.858621 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:39.858679 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:39.858853 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:39.859572 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:39.860776 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:39.861952 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:39.862527 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:39.862649 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:39.865282 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:39.865286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:39.865286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:39.865298 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:39.867100 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:39.867458 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:39.867460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:39.867461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:39.867620 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:39.867813 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:39.867813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:39.867813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:39.868086 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:39.868242 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:39.868738 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 21:08:39.868874 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260812 21:08:39.872792 16629 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:39.873425 16629 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:39.877327 16629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7135046166a14b7f8e37f6778d699aa4: Generated new TSK 0
I20260812 21:08:39.877480 16629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:39.907430 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43967
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:39.990513 16647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:39.990737 16647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:39.990769 16647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:39.991932 16647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:39.991973 16647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:39.992008 16647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:39.993583 16647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:39.993818 16647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:39.994024 16647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:39.999011 16657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:39.999068 16652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:39.999157 16647 server_base.cc:1034] running on GCE node
W20260812 21:08:39.999094 16653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:39.999527 16647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:39.999778 16647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:40.000941 16647 hybrid_clock.cc:648] HybridClock initialized: now 1786568920000922 us; error 34 us; skew 500 ppm
Aug 12 21:08:39 dist-test-slave-lkx2 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568919, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:40.003393 16647 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:40.003831 16647 webserver.cc:460] Webserver started at http://127.11.207.129:37675/ using document root <none> and password file <none>
I20260812 21:08:40.004040 16647 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:40.004081 16647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:40.004209 16647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:40.005306 16647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "28aefdaafbbd40959ad7e83805d28d74"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "d1607e0986cbff9d1f8c48e5e28879e8"
server_key_iv: "c342fe866a14ce89d85ec06e6e0940b4"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.005627 16647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "28aefdaafbbd40959ad7e83805d28d74"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "d1607e0986cbff9d1f8c48e5e28879e8"
server_key_iv: "c342fe866a14ce89d85ec06e6e0940b4"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.006858 16647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:40.007529 16680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.007694 16647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:40.007777 16647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "28aefdaafbbd40959ad7e83805d28d74"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "d1607e0986cbff9d1f8c48e5e28879e8"
server_key_iv: "c342fe866a14ce89d85ec06e6e0940b4"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.007833 16647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:40.023540 16647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:40.025781 16647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:40.025902 16647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:40.026103 16647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:40.026443 16647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:40.026479 16647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.026500 16647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:40.026523 16647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.032328 16647 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:44131
I20260812 21:08:40.032428 16870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:44131 every 8 connection(s)
I20260812 21:08:40.032727 16647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:40.033435 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16647
I20260812 21:08:40.033531 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:40.033732 12094 external_mini_cluster.cc:1428] Setting key fb4a5423ace1d5b735a662cfc8a253c2
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[16480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568919, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
I20260812 21:08:40.043154 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:40.033678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:59875 (local address 127.11.207.190:43967)
0812 21:08:40.033738 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:40.033740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:40.033944 (+ 204us) server_negotiation.cc:407] Connection header received
0812 21:08:40.034592 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:40.034593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:40.034608 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:40.034670 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:40.035639 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.036329 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.037816 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.038103 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.038172 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:40.040501 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:40.040502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:40.040503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:40.040511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:40.041810 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:40.042194 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:40.042195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:40.042195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:40.042250 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:40.042476 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:40.042477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:40.042477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:40.042588 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:40.042646 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:40.042952 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 21:08:40.043056 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:08:40.043311 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:40.033597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43967 (local address 127.11.207.129:59875)
0812 21:08:40.033784 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:40.033838 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:40.034489 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:40.034743 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:40.034748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:40.035347 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:40.035554 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:40.035560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.036463 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.036464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:40.037730 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:40.037733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.038202 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.038203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:40.038312 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:40.038889 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:40.038899 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:40.040406 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:40.041892 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:40.041900 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:40.041902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:40.042125 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:40.042304 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:40.042305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:40.042305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:40.042417 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:40.042646 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:40.042654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:40.042819 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 21:08:40.042918 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 21:08:40.043089 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260812 21:08:40.043864 16871 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43967
I20260812 21:08:40.043972 16871 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:40.044166 16871 heartbeater.cc:511] Master 127.11.207.190:43967 requested a full tablet report, sending...
I20260812 21:08:40.044797 16550 ts_manager.cc:194] Registered new tserver with Master: 28aefdaafbbd40959ad7e83805d28d74 (127.11.207.129:44131)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
I20260812 21:08:40.045572 16550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:59875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:40.061008 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43967
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:40.144140 16881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:40.144388 16881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:40.144414 16881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:40.145874 16881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:40.145929 16881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:40.145969 16881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:40.147467 16881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:40.147675 16881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:40.147862 16881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:40.152761 16889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:40.152817 16890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:40.152976 16881 server_base.cc:1034] running on GCE node
W20260812 21:08:40.153054 16892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:40.153286 16881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:40.153530 16881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:40.154721 16881 hybrid_clock.cc:648] HybridClock initialized: now 1786568920154704 us; error 31 us; skew 500 ppm
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568920, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:40.157259 16881 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:40.157809 16881 webserver.cc:460] Webserver started at http://127.11.207.130:42843/ using document root <none> and password file <none>
I20260812 21:08:40.158038 16881 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:40.158087 16881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:40.158219 16881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:40.159273 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "10c28f593d854eb5a12a617bc61e5995"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "8b2087ec5cf23a4843a252b030f32542"
server_key_iv: "ba27e5ac712c20e5106b99c723a0dfb7"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.159618 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "10c28f593d854eb5a12a617bc61e5995"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "8b2087ec5cf23a4843a252b030f32542"
server_key_iv: "ba27e5ac712c20e5106b99c723a0dfb7"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.161036 16881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:40.161751 16902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.161903 16881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:40.161993 16881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "10c28f593d854eb5a12a617bc61e5995"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "8b2087ec5cf23a4843a252b030f32542"
server_key_iv: "ba27e5ac712c20e5106b99c723a0dfb7"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.162052 16881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:40.168293 16881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:40.169936 16881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:40.170100 16881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:40.170305 16881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:40.170593 16881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:40.170625 16881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.170652 16881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:40.170675 16881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.176491 16881 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:46701
I20260812 21:08:40.176553 17103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:46701 every 8 connection(s)
I20260812 21:08:40.176870 16881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[16480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568920, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:40.185436 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:40.177571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:50143 (local address 127.11.207.190:43967)
0812 21:08:40.177626 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:40.177628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:40.177836 (+ 208us) server_negotiation.cc:407] Connection header received
0812 21:08:40.178315 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:40.178315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:40.178324 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:40.178386 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:40.179294 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.179807 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.180806 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.181005 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.181059 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:40.183084 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:40.183086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:40.183086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:40.183093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:40.184325 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:40.184697 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:40.184697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:40.184698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:40.184748 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:40.184934 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:40.184934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:40.184935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:40.185036 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:40.185082 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:40.185291 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 21:08:40.185370 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:08:40.185611 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:40.177540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43967 (local address 127.11.207.130:50143)
0812 21:08:40.177707 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:40.177750 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:40.178237 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:40.178442 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:40.178445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:40.179021 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:40.179222 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:40.179227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.179883 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.179883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:40.180730 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:40.180733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.181073 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.181073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:40.181174 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:40.181615 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:40.181624 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:40.182995 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:40.184411 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:40.184418 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:40.184420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:40.184632 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:40.184795 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:40.184796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:40.184796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:40.184892 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:40.185078 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:40.185081 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:40.185192 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 21:08:40.185276 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:08:40.185403 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260812 21:08:40.186093 17104 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43967
I20260812 21:08:40.186192 17104 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:40.186371 17104 heartbeater.cc:511] Master 127.11.207.190:43967 requested a full tablet report, sending...
I20260812 21:08:40.186573 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16881
I20260812 21:08:40.186676 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:40.186790 16550 ts_manager.cc:194] Registered new tserver with Master: 10c28f593d854eb5a12a617bc61e5995 (127.11.207.130:46701)
I20260812 21:08:40.186880 12094 external_mini_cluster.cc:1428] Setting key a10aadc676d810626988789a1ad90f68
I20260812 21:08:40.187372 16550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:50143
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:40.213325 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43967
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:40.296167 17111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:40.296388 17111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:40.296411 17111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:40.297668 17111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:40.297721 17111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:40.297747 17111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:40.299196 17111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:40.299396 17111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:40.299574 17111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:40.304394 17123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:40.304420 17125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:40.304553 17111 server_base.cc:1034] running on GCE node
W20260812 21:08:40.304420 17120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:40.304816 17111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:40.305025 17111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:40.306175 17111 hybrid_clock.cc:648] HybridClock initialized: now 1786568920306159 us; error 32 us; skew 500 ppm
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568920, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:40.308555 17111 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:40.308948 17111 webserver.cc:460] Webserver started at http://127.11.207.131:36509/ using document root <none> and password file <none>
I20260812 21:08:40.309151 17111 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:40.309198 17111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:40.309332 17111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:40.310338 17111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "bce47e4801c94d8e863c70aa81c0e8af"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "c4f9b73511132f5c8f344f21baa3986b"
server_key_iv: "6ce63944f2a01122982174c3069ecfeb"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.310706 17111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "bce47e4801c94d8e863c70aa81c0e8af"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "c4f9b73511132f5c8f344f21baa3986b"
server_key_iv: "6ce63944f2a01122982174c3069ecfeb"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.311934 17111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:08:40.312620 17140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.312762 17111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.312846 17111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "bce47e4801c94d8e863c70aa81c0e8af"
format_stamp: "Formatted at 2026-08-12 21:08:40 on dist-test-slave-lkx2"
server_key: "c4f9b73511132f5c8f344f21baa3986b"
server_key_iv: "6ce63944f2a01122982174c3069ecfeb"
server_key_version: "encryptionkey@0"
I20260812 21:08:40.312907 17111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:40.340408 17111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:40.342232 17111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:40.342372 17111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:40.342603 17111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:40.342927 17111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:40.342968 17111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.342998 17111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:40.343015 17111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:40.348698 17111 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:38727
I20260812 21:08:40.348769 17336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:38727 every 8 connection(s)
I20260812 21:08:40.349061 17111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[16480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568920, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:40.357867 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:40.349730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:54785 (local address 127.11.207.190:43967)
0812 21:08:40.349785 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:40.349787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:40.349985 (+ 198us) server_negotiation.cc:407] Connection header received
0812 21:08:40.350489 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:40.350490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:40.350503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:40.350555 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:40.351525 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.352036 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.353050 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.353248 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.353303 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:40.355328 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:40.355329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:40.355330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:40.355337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:40.356609 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:40.356988 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:40.356990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:40.356990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:40.357045 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:40.357261 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:40.357261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:40.357262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:40.357375 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:40.357432 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:40.357694 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 21:08:40.357776 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:08:40.358036 17345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:40.349704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43967 (local address 127.11.207.131:54785)
0812 21:08:40.349868 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:40.349907 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:40.350392 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:40.350623 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:40.350627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:40.351206 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:40.351407 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:40.351411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.352121 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.352122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:40.352966 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:40.352969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:40.353316 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:40.353317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:40.353453 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:40.353920 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:40.353929 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:40.355241 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:40.356682 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:40.356689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:40.356691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:40.356921 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:40.357099 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:40.357099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:40.357100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:40.357205 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:40.357430 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:40.357437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:40.357571 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 21:08:40.357658 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:08:40.357813 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260812 21:08:40.358500 17339 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43967
I20260812 21:08:40.358594 17339 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:40.358767 17339 heartbeater.cc:511] Master 127.11.207.190:43967 requested a full tablet report, sending...
I20260812 21:08:40.359139 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17111
I20260812 21:08:40.359228 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:40.359368 16550 ts_manager.cc:194] Registered new tserver with Master: bce47e4801c94d8e863c70aa81c0e8af (127.11.207.131:38727)
I20260812 21:08:40.359409 12094 external_mini_cluster.cc:1428] Setting key eed39d1f3b390576a51e650b9089b241
I20260812 21:08:40.359834 16550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:54785
I20260812 21:08:40.360173 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 21:08:40.372450 16608 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:40.371513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.11.207.190:43967)
0812 21:08:40.371574 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:40.371577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:40.371670 (+ 93us) server_negotiation.cc:407] Connection header received
0812 21:08:40.371745 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:40.371746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:40.371756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:40.371802 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:40.372327 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52768: BlockingRecv error: recv got EOF from 127.0.0.1:52768 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:08:40.373212 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16647
I20260812 21:08:40.400987 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16881
I20260812 21:08:40.420010 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17111
I20260812 21:08:40.429983 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16501
2026-08-12T21:08:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2034 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[17361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[17361](info): set up 2 sockets
Aug 12 21:08:40 dist-test-slave-lkx2 krb5kdc[17361](info): commencing operation
krb5kdc: starting...
W20260812 21:08:41.632771 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:41 dist-test-slave-lkx2 krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:41.752614 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:32797
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:32797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:41.836907 17384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:41.837143 17384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:41.837178 17384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:41.838352 17384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:41.838394 17384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:41.838407 17384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:41.838418 17384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:41.838428 17384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:41.839938 17384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:32797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:32797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:41.840157 17384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:41.840413 17384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:41.845688 17400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:41.845748 17384 server_base.cc:1034] running on GCE node
W20260812 21:08:41.845688 17398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:41.845714 17394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:41.846092 17384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:41.846415 17384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:41.847579 17384 hybrid_clock.cc:648] HybridClock initialized: now 1786568921847563 us; error 32 us; skew 500 ppm
Aug 12 21:08:41 dist-test-slave-lkx2 krb5kdc[17361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568921, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:41.856040 17384 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:41.856544 17384 webserver.cc:460] Webserver started at http://127.11.207.190:41953/ using document root <none> and password file <none>
I20260812 21:08:41.856776 17384 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:41.856822 17384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:41.856957 17384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:41.858028 17384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f"
format_stamp: "Formatted at 2026-08-12 21:08:41 on dist-test-slave-lkx2"
server_key: "16de570dbe8c813f4d5ab73d64d2719d"
server_key_iv: "738284a118d13ee57449c96f40cfe03d"
server_key_version: "encryptionkey@0"
I20260812 21:08:41.858345 17384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f"
format_stamp: "Formatted at 2026-08-12 21:08:41 on dist-test-slave-lkx2"
server_key: "16de570dbe8c813f4d5ab73d64d2719d"
server_key_iv: "738284a118d13ee57449c96f40cfe03d"
server_key_version: "encryptionkey@0"
I20260812 21:08:41.859567 17384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:41.860641 17412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:41.860867 17384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:41.860966 17384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f"
format_stamp: "Formatted at 2026-08-12 21:08:41 on dist-test-slave-lkx2"
server_key: "16de570dbe8c813f4d5ab73d64d2719d"
server_key_iv: "738284a118d13ee57449c96f40cfe03d"
server_key_version: "encryptionkey@0"
I20260812 21:08:41.861040 17384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:41.874436 17384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:41.876157 17384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:41.876266 17384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:41.880075 17384 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:32797
I20260812 21:08:41.880110 17491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:32797 every 8 connection(s)
I20260812 21:08:41.880652 17384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:41.881301 17493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:41.883556 17493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f: Bootstrap starting.
I20260812 21:08:41.884130 17493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:41.884431 17493 log.cc:824] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:41.885077 17493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f: No bootstrap required, opened a new log
I20260812 21:08:41.886626 17493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 32797 } }
I20260812 21:08:41.886761 17493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:41.886793 17493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fef8fc8a5542d2bc8fd37ea3c8ba9f, State: Initialized, Role: FOLLOWER
I20260812 21:08:41.886938 17493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 32797 } }
I20260812 21:08:41.887017 17493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:41.887056 17493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:41.887101 17493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:41.887873 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17384
I20260812 21:08:41.887954 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:41.888048 17493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 32797 } }
I20260812 21:08:41.888120 12094 external_mini_cluster.cc:1428] Setting key 3cf47d2794a6ab1567709d174ef85bb7
I20260812 21:08:41.888149 17493 leader_election.cc:302] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44fef8fc8a5542d2bc8fd37ea3c8ba9f; no voters:
I20260812 21:08:41.888314 17493 leader_election.cc:288] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:41.888388 17497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:41.888510 17497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [term 1 LEADER]: Becoming Leader. State: Replica: 44fef8fc8a5542d2bc8fd37ea3c8ba9f, State: Running, Role: LEADER
I20260812 21:08:41.888595 17493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:41.888586 17497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 32797 } }
I20260812 21:08:41.888875 17497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 32797 } } }
I20260812 21:08:41.888945 17497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:41.888972 17498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44fef8fc8a5542d2bc8fd37ea3c8ba9f. Latest consensus state: current_term: 1 leader_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fef8fc8a5542d2bc8fd37ea3c8ba9f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 32797 } } }
I20260812 21:08:41.889068 17498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:41.889822 17509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:41.890429 17509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:41.892437 17509 catalog_manager.cc:1347] Generated new cluster ID: fd86d30d377c4497ba2c462d0b178d94
I20260812 21:08:41.892522 17509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:41 dist-test-slave-lkx2 krb5kdc[17361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:41.898178 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:41.888642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.11.207.190:32797)
0812 21:08:41.889058 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:41.889113 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:41.889166 (+ 53us) server_negotiation.cc:407] Connection header received
0812 21:08:41.889288 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:41.889296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:41.889354 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:41.889506 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:41.890304 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:41.891509 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:41.892423 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:41.892930 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:41.893029 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:41.895217 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:41.895222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:41.895222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:41.895234 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:41.896808 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:41.897124 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:41.897126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:41.897127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:41.897224 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:41.897396 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:41.897397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:41.897397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:41.897616 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:41.897775 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:41.897848 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 21:08:41.897980 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":72,"threads_started":1}
I20260812 21:08:41.898995 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:41.912946 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:41.909792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.11.207.190:32797)
0812 21:08:41.909847 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:41.909848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:41.909968 (+ 120us) server_negotiation.cc:407] Connection header received
0812 21:08:41.910027 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:41.910028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:41.910040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:41.910085 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:41.910438 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:41.910898 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:41.911605 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:41.911807 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:41.911861 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:41.912519 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:41.912520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:41.912520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:41.912711 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:41.912727 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:41.912733 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:41.912780 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:41.912844 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 21:08:41.912867 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:08:41.914714 17509 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:41.915196 17509 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:41.927016 17509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44fef8fc8a5542d2bc8fd37ea3c8ba9f: Generated new TSK 0
I20260812 21:08:41.927212 17509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:41.934654 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17384
2026-08-12T21:08:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1520 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:42 dist-test-slave-lkx2 krb5kdc[17543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:42 dist-test-slave-lkx2 krb5kdc[17543](info): set up 2 sockets
Aug 12 21:08:42 dist-test-slave-lkx2 krb5kdc[17543](info): commencing operation
krb5kdc: starting...
W20260812 21:08:43.168438 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:43.291213 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:36475
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:36475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:43.374192 17565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:43.374419 17565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:43.374441 17565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:43.375540 17565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:43.375576 17565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:43.375588 17565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:43.375599 17565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:43.375610 17565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:43.377085 17565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:36475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:36475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:43.377286 17565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:43.377471 17565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:43.382248 17573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:43.382390 17565 server_base.cc:1034] running on GCE node
W20260812 21:08:43.382258 17572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:43.382253 17579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:43.382793 17565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:43.383122 17565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:43.384281 17565 hybrid_clock.cc:648] HybridClock initialized: now 1786568923384261 us; error 33 us; skew 500 ppm
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:43.386880 17565 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:43.387355 17565 webserver.cc:460] Webserver started at http://127.11.207.190:41671/ using document root <none> and password file <none>
I20260812 21:08:43.387562 17565 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:43.387611 17565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:43.387750 17565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:43.388897 17565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "d811ed1287c34a1a8107acfd50eefb1a"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "855651b3c348c32d5aba637204ec56c3"
server_key_iv: "0c5bce85cca2c33e287a5d9c1dc4a435"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.389240 17565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "d811ed1287c34a1a8107acfd50eefb1a"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "855651b3c348c32d5aba637204ec56c3"
server_key_iv: "0c5bce85cca2c33e287a5d9c1dc4a435"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.390570 17565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:43.391320 17589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.391490 17565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:43.391577 17565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "d811ed1287c34a1a8107acfd50eefb1a"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "855651b3c348c32d5aba637204ec56c3"
server_key_iv: "0c5bce85cca2c33e287a5d9c1dc4a435"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.391639 17565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:43.426615 17565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:43.428925 17565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:43.429072 17565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:43.433025 17565 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:36475
I20260812 21:08:43.433076 17684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:36475 every 8 connection(s)
I20260812 21:08:43.433435 17565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:43.434667 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:43.437095 17685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a: Bootstrap starting.
I20260812 21:08:43.437168 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17565
I20260812 21:08:43.437266 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:43.437455 12094 external_mini_cluster.cc:1428] Setting key af7c7b99e962e907709049582ec67ce9
I20260812 21:08:43.437835 17685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:43.438184 17685 log.cc:824] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:43.438882 17685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a: No bootstrap required, opened a new log
I20260812 21:08:43.441084 17685 raft_consensus.cc:348] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d811ed1287c34a1a8107acfd50eefb1a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36475 } }
I20260812 21:08:43.441275 17685 raft_consensus.cc:374] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:43.441313 17685 raft_consensus.cc:729] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d811ed1287c34a1a8107acfd50eefb1a, State: Initialized, Role: FOLLOWER
I20260812 21:08:43.441437 17685 consensus_queue.cc:260] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d811ed1287c34a1a8107acfd50eefb1a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36475 } }
I20260812 21:08:43.441519 17685 raft_consensus.cc:388] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:43.441552 17685 raft_consensus.cc:482] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:43.441593 17685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:43.442170 17685 raft_consensus.cc:504] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d811ed1287c34a1a8107acfd50eefb1a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36475 } }
I20260812 21:08:43.442279 17685 leader_election.cc:302] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d811ed1287c34a1a8107acfd50eefb1a; no voters:
I20260812 21:08:43.442447 17685 leader_election.cc:288] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:43.442466 17695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:43.442628 17695 raft_consensus.cc:686] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [term 1 LEADER]: Becoming Leader. State: Replica: d811ed1287c34a1a8107acfd50eefb1a, State: Running, Role: LEADER
I20260812 21:08:43.442716 17685 sys_catalog.cc:564] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:43.442728 17695 consensus_queue.cc:237] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d811ed1287c34a1a8107acfd50eefb1a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36475 } }
I20260812 21:08:43.444242 17696 sys_catalog.cc:455] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d811ed1287c34a1a8107acfd50eefb1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d811ed1287c34a1a8107acfd50eefb1a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36475 } } }
I20260812 21:08:43.444329 17696 sys_catalog.cc:458] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:43.444535 17697 sys_catalog.cc:455] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d811ed1287c34a1a8107acfd50eefb1a. Latest consensus state: current_term: 1 leader_uuid: "d811ed1287c34a1a8107acfd50eefb1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d811ed1287c34a1a8107acfd50eefb1a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36475 } } }
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:43.444602 17697 sys_catalog.cc:458] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a [sys.catalog]: This master's current role is: LEADER
W20260812 21:08:43.444649 17720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:43.444717 17720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:43.444769 17722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:43.444939 17722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:43.446664 17722 catalog_manager.cc:1347] Generated new cluster ID: d2ac0e8f80c54618877f36b7dcde2093
I20260812 21:08:43.446727 17722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:43.448726 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.437988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.11.207.190:36475)
0812 21:08:43.438189 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:43.438199 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:43.438226 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:08:43.438792 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:43.438801 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:43.438854 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:43.439002 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:43.439792 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.440709 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.441757 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.442162 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.442255 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.445145 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:43.445149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:43.445149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:43.445161 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:43.447118 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.447521 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.447524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.447524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.447619 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.447818 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.447819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.447819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.448095 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:43.448246 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:43.448394 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 21:08:43.448523 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260812 21:08:43.459187 17722 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:43.459930 17722 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:43.464511 17722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d811ed1287c34a1a8107acfd50eefb1a: Generated new TSK 0
I20260812 21:08:43.464701 17722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:43.494952 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36475
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:43.578114 17729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:43.578330 17729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:43.578354 17729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:43.579541 17729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:43.579584 17729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:43.579612 17729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:43.581112 17729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:43.581346 17729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:43.581566 17729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:43.586473 17741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:43.586542 17736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:43.586509 17735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:43.586581 17729 server_base.cc:1034] running on GCE node
I20260812 21:08:43.586885 17729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:43.587117 17729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:43.588267 17729 hybrid_clock.cc:648] HybridClock initialized: now 1786568923588241 us; error 36 us; skew 500 ppm
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:43.590853 17729 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:43.591300 17729 webserver.cc:460] Webserver started at http://127.11.207.129:45001/ using document root <none> and password file <none>
I20260812 21:08:43.591480 17729 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:43.591517 17729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:43.591609 17729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:43.592658 17729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "3b178706f82147ab921790ebaecf7c04"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "21f5f70d78033ba95c3b5cfd7ea67abe"
server_key_iv: "584d8fbd363b4ab719ebd93a9508d12e"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.593070 17729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "3b178706f82147ab921790ebaecf7c04"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "21f5f70d78033ba95c3b5cfd7ea67abe"
server_key_iv: "584d8fbd363b4ab719ebd93a9508d12e"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.594485 17729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:43.595187 17754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.595407 17729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:43.595489 17729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "3b178706f82147ab921790ebaecf7c04"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "21f5f70d78033ba95c3b5cfd7ea67abe"
server_key_iv: "584d8fbd363b4ab719ebd93a9508d12e"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.595543 17729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:43.608229 17729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:43.610260 17729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:43.610373 17729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:43.610564 17729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:43.610877 17729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:43.610910 17729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.610931 17729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:43.610946 17729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.616519 17729 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:42699
I20260812 21:08:43.616595 17948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:42699 every 8 connection(s)
I20260812 21:08:43.616904 17729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:43.620093 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17729
I20260812 21:08:43.620211 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:43.620453 12094 external_mini_cluster.cc:1428] Setting key 0bdfdd2752291183761176d7548c5094
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:43.627141 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.617848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:53661 (local address 127.11.207.190:36475)
0812 21:08:43.617939 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:43.617941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:43.618073 (+ 132us) server_negotiation.cc:407] Connection header received
0812 21:08:43.618598 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:43.618599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:43.618611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:43.618667 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:43.619613 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.620166 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.621390 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.621599 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.621655 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.624017 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:43.624019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:43.624019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:43.624027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:43.625568 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.626065 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.626067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.626067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.626135 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.626388 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.626389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.626389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.626536 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:43.626610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:43.626944 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 21:08:43.627043 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 21:08:43.627238 17951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.617749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:36475 (local address 127.11.207.129:53661)
0812 21:08:43.617944 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:43.617989 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:43.618499 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:43.618736 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:43.618739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:43.619312 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:43.619514 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:43.619518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.620257 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.620259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:43.621306 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:43.621309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.621676 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.621676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:43.621814 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.622335 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:43.622343 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:43.623890 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:43.625671 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:43.625679 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:43.625682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:43.625962 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:43.626217 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:43.626218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:43.626218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:43.626330 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:43.626661 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:43.626669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:43.626804 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:08:43.626906 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 21:08:43.627042 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260812 21:08:43.627831 17949 heartbeater.cc:347] Connected to a master server at 127.11.207.190:36475
I20260812 21:08:43.627938 17949 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:43.628113 17949 heartbeater.cc:511] Master 127.11.207.190:36475 requested a full tablet report, sending...
I20260812 21:08:43.628742 17618 ts_manager.cc:194] Registered new tserver with Master: 3b178706f82147ab921790ebaecf7c04 (127.11.207.129:42699)
I20260812 21:08:43.629648 17618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:53661
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:43.647722 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36475
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:43.730661 17958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:43.730880 17958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:43.730901 17958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:43.732013 17958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:43.732049 17958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:43.732075 17958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:43.733620 17958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:43.733841 17958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:43.734026 17958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:43.739125 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:43.739221 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:43.739183 17958 server_base.cc:1034] running on GCE node
W20260812 21:08:43.739151 17976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:43.739617 17958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:43.739889 17958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:43.741037 17958 hybrid_clock.cc:648] HybridClock initialized: now 1786568923741025 us; error 32 us; skew 500 ppm
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:43.743599 17958 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:43.744097 17958 webserver.cc:460] Webserver started at http://127.11.207.130:35215/ using document root <none> and password file <none>
I20260812 21:08:43.744321 17958 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:43.744398 17958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:43.744534 17958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:43.745576 17958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "58c0bde5c33a4664be6274e98209b75c"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "36447da3fc1d2e68860978da1fe7d039"
server_key_iv: "3d1c0d6bc2a895f81cfbbea99e51bb18"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.745930 17958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "58c0bde5c33a4664be6274e98209b75c"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "36447da3fc1d2e68860978da1fe7d039"
server_key_iv: "3d1c0d6bc2a895f81cfbbea99e51bb18"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.747274 17958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:43.748055 17987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.748216 17958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:43.748307 17958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "58c0bde5c33a4664be6274e98209b75c"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "36447da3fc1d2e68860978da1fe7d039"
server_key_iv: "3d1c0d6bc2a895f81cfbbea99e51bb18"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.748405 17958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:43.759809 17958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:43.762171 17958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:43.762321 17958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:43.762558 17958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:43.762877 17958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:43.762910 17958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.762940 17958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:43.762964 17958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.768690 17958 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:34435
I20260812 21:08:43.768817 18191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:34435 every 8 connection(s)
I20260812 21:08:43.769074 17958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:43.773263 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17958
I20260812 21:08:43.773371 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:43.773602 12094 external_mini_cluster.cc:1428] Setting key 1c6e5789d6370442ac2352f035cdfa13
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:43.778307 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.769780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:55871 (local address 127.11.207.190:36475)
0812 21:08:43.769838 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:43.769840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:43.770057 (+ 217us) server_negotiation.cc:407] Connection header received
0812 21:08:43.770547 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:43.770548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:43.770558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:43.770611 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:43.771560 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.772058 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.773078 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.773273 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.773331 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.775621 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:43.775623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:43.775623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:43.775631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:43.777018 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.777414 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.777415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.777416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.777471 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.777691 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.777692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.777693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.777810 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:43.777868 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:43.778134 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 21:08:43.778216 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:08:43.778496 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.769759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:36475 (local address 127.11.207.130:55871)
0812 21:08:43.769930 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:43.769974 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:43.770474 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:43.770679 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:43.770682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:43.771284 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:43.771486 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:43.771491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.772122 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.772123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:43.773000 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:43.773004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.773360 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.773361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:43.773525 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.774084 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:43.774095 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:43.775526 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:43.777112 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:43.777120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:43.777122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:43.777348 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:43.777525 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:43.777526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:43.777527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:43.777637 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:43.777866 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:43.777873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:43.778013 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 21:08:43.778102 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:08:43.778275 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260812 21:08:43.779006 18193 heartbeater.cc:347] Connected to a master server at 127.11.207.190:36475
I20260812 21:08:43.779109 18193 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:43.779317 18193 heartbeater.cc:511] Master 127.11.207.190:36475 requested a full tablet report, sending...
I20260812 21:08:43.780472 17618 ts_manager.cc:194] Registered new tserver with Master: 58c0bde5c33a4664be6274e98209b75c (127.11.207.130:34435)
I20260812 21:08:43.780987 17618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:55871
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:43.801461 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36475
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:43.886057 18201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:43.886276 18201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:43.886300 18201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:43.887434 18201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:43.887477 18201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:43.887502 18201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:43.889039 18201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:43.889271 18201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:43.889499 18201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:43.894407 18213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:43.894408 18207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:43.894438 18209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:43.894634 18201 server_base.cc:1034] running on GCE node
I20260812 21:08:43.894868 18201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:43.895077 18201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:43.896245 18201 hybrid_clock.cc:648] HybridClock initialized: now 1786568923896235 us; error 30 us; skew 500 ppm
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:43.898819 18201 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:43.899222 18201 webserver.cc:460] Webserver started at http://127.11.207.131:34187/ using document root <none> and password file <none>
I20260812 21:08:43.899397 18201 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:43.899442 18201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:43.899538 18201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:43.900593 18201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "39890940a1eb4b238bc6716adb5927ad"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "1ccdb84ffaf442b9b1f3f6eb91f6354e"
server_key_iv: "4ef1fa7abcf0c95b571da47ce41c4b48"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.900911 18201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "39890940a1eb4b238bc6716adb5927ad"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "1ccdb84ffaf442b9b1f3f6eb91f6354e"
server_key_iv: "4ef1fa7abcf0c95b571da47ce41c4b48"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.902163 18201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:43.902908 18221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.903088 18201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:43.903187 18201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "39890940a1eb4b238bc6716adb5927ad"
format_stamp: "Formatted at 2026-08-12 21:08:43 on dist-test-slave-lkx2"
server_key: "1ccdb84ffaf442b9b1f3f6eb91f6354e"
server_key_iv: "4ef1fa7abcf0c95b571da47ce41c4b48"
server_key_version: "encryptionkey@0"
I20260812 21:08:43.903247 18201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:43.917013 18201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:43.919287 18201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:43.919443 18201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:43.919672 18201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:43.920001 18201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:43.920035 18201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.920065 18201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:43.920089 18201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:43.925886 18201 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:40559
I20260812 21:08:43.925953 18424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:40559 every 8 connection(s)
I20260812 21:08:43.926265 18201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:43.927340 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18201
I20260812 21:08:43.927433 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:43.927620 12094 external_mini_cluster.cc:1428] Setting key 36e79265d0de68939bd9dcc1bbdc1f64
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:43.935343 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.926983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37173 (local address 127.11.207.190:36475)
0812 21:08:43.927068 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:43.927071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:43.927270 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:08:43.927872 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:43.927873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:43.927887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:43.927945 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:43.929030 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.929543 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.930535 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.930746 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.930803 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.932898 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:43.932900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:43.932900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:43.932908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:43.934062 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.934450 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.934451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.934452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.934510 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.934728 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.934729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.934729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.934847 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:43.934906 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:43.935166 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 21:08:43.935251 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:08:43.935559 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.926946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:36475 (local address 127.11.207.131:37173)
0812 21:08:43.927138 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:43.927177 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:43.927753 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:43.928069 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:43.928073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:43.928727 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:43.928934 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:43.928939 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.929630 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.929631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:43.930451 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:43.930454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.930819 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.930820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:43.930948 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.931443 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:43.931452 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:43.932808 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:43.934140 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:43.934148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:43.934150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:43.934382 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:43.934564 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:43.934565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:43.934565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:43.934670 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:43.934902 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:43.934909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:43.935047 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 21:08:43.935138 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:43.935317 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
I20260812 21:08:43.936004 18425 heartbeater.cc:347] Connected to a master server at 127.11.207.190:36475
I20260812 21:08:43.936103 18425 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:43.936286 18425 heartbeater.cc:511] Master 127.11.207.190:36475 requested a full tablet report, sending...
I20260812 21:08:43.936743 17618 ts_manager.cc:194] Registered new tserver with Master: 39890940a1eb4b238bc6716adb5927ad (127.11.207.131:40559)
I20260812 21:08:43.937124 17618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:37173
I20260812 21:08:43.939327 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 21:08:43 dist-test-slave-lkx2 krb5kdc[17543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568923, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:43.962182 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:43.955793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.11.207.190:36475)
0812 21:08:43.955854 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:43.955857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:43.955939 (+ 82us) server_negotiation.cc:407] Connection header received
0812 21:08:43.955998 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:43.955999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:43.956008 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:43.956057 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:43.956614 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.957114 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.957845 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:43.958048 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:43.958112 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:43.960189 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:43.960191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:43.960191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:43.960199 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:43.961300 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.961576 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.961577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.961577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.961621 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:43.961760 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:43.961760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:43.961760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:43.961892 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:43.961963 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:43.962030 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 21:08:43.962092 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 21:08:43.962438 17618 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36890
I20260812 21:08:43.962846 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17729
I20260812 21:08:43.985407 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17958
I20260812 21:08:43.999330 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18201
I20260812 21:08:44.010845 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17565
2026-08-12T21:08:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2059 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:44 dist-test-slave-lkx2 krb5kdc[18447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:44 dist-test-slave-lkx2 krb5kdc[18447](info): set up 2 sockets
Aug 12 21:08:44 dist-test-slave-lkx2 krb5kdc[18447](info): commencing operation
krb5kdc: starting...
W20260812 21:08:45.210314 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:45.334590 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:40481
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:40481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:45.417105 18469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:45.417327 18469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:45.417358 18469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:45.418514 18469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:45.418560 18469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:45.418578 18469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:45.418596 18469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:45.418610 18469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:45.420173 18469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:40481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:40481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:45.420456 18469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:45.420674 18469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:45.425390 18479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.425416 18489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:45.425498 18469 server_base.cc:1034] running on GCE node
W20260812 21:08:45.425407 18487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:45.425895 18469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:45.426182 18469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:45.427346 18469 hybrid_clock.cc:648] HybridClock initialized: now 1786568925427323 us; error 36 us; skew 500 ppm
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:45.430154 18469 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:45.430616 18469 webserver.cc:460] Webserver started at http://127.11.207.190:44763/ using document root <none> and password file <none>
I20260812 21:08:45.430846 18469 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:45.430892 18469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:45.431007 18469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:45.432189 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "b894eb7ba1864573ba5dd1282116493c"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "d5eb7fbc2dff92f3c0370c7b4a3b47e4"
server_key_iv: "8951b6766007af35479fc98cca5f15f2"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.432588 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "b894eb7ba1864573ba5dd1282116493c"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "d5eb7fbc2dff92f3c0370c7b4a3b47e4"
server_key_iv: "8951b6766007af35479fc98cca5f15f2"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.433916 18469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:45.434600 18499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.434739 18469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:45.434827 18469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "b894eb7ba1864573ba5dd1282116493c"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "d5eb7fbc2dff92f3c0370c7b4a3b47e4"
server_key_iv: "8951b6766007af35479fc98cca5f15f2"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.434890 18469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:45.475865 18469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:45.477969 18469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:45.478132 18469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:45.482312 18469 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:40481
I20260812 21:08:45.482362 18589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:40481 every 8 connection(s)
I20260812 21:08:45.482859 18469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:45.483722 18590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:45.486176 18590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c: Bootstrap starting.
I20260812 21:08:45.486792 18590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:45.487038 18590 log.cc:824] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:45.487764 18590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c: No bootstrap required, opened a new log
I20260812 21:08:45.489434 18590 raft_consensus.cc:348] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b894eb7ba1864573ba5dd1282116493c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40481 } }
I20260812 21:08:45.489600 18590 raft_consensus.cc:374] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:45.489627 18590 raft_consensus.cc:729] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b894eb7ba1864573ba5dd1282116493c, State: Initialized, Role: FOLLOWER
I20260812 21:08:45.489744 18590 consensus_queue.cc:260] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b894eb7ba1864573ba5dd1282116493c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40481 } }
I20260812 21:08:45.489823 18590 raft_consensus.cc:388] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:45.489856 18590 raft_consensus.cc:482] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:45.489888 18590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:45.490772 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18469
I20260812 21:08:45.490871 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:45.490862 18590 raft_consensus.cc:504] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b894eb7ba1864573ba5dd1282116493c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40481 } }
I20260812 21:08:45.490959 18590 leader_election.cc:302] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b894eb7ba1864573ba5dd1282116493c; no voters:
I20260812 21:08:45.491065 12094 external_mini_cluster.cc:1428] Setting key ffc1559607d5b8d9ea1d265160116dce
I20260812 21:08:45.491132 18590 leader_election.cc:288] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:45.491183 18595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:45.491370 18595 raft_consensus.cc:686] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [term 1 LEADER]: Becoming Leader. State: Replica: b894eb7ba1864573ba5dd1282116493c, State: Running, Role: LEADER
I20260812 21:08:45.491405 18590 sys_catalog.cc:564] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:45.491472 18595 consensus_queue.cc:237] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b894eb7ba1864573ba5dd1282116493c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40481 } }
I20260812 21:08:45.491896 18597 sys_catalog.cc:455] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b894eb7ba1864573ba5dd1282116493c. Latest consensus state: current_term: 1 leader_uuid: "b894eb7ba1864573ba5dd1282116493c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b894eb7ba1864573ba5dd1282116493c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40481 } } }
I20260812 21:08:45.491909 18596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b894eb7ba1864573ba5dd1282116493c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b894eb7ba1864573ba5dd1282116493c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40481 } } }
I20260812 21:08:45.492079 18597 sys_catalog.cc:458] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:45.492089 18596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:45.492558 18618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:45.493117 18618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:45.494855 18618 catalog_manager.cc:1347] Generated new cluster ID: 779900d6bfaf4f7f8f55689cbd4c14f3
I20260812 21:08:45.494910 18618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:45.500921 18618 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:45.501608 18618 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:45.501825 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:45.491634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44002 (local address 127.11.207.190:40481)
0812 21:08:45.492577 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:45.492588 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:45.492627 (+ 39us) server_negotiation.cc:407] Connection header received
0812 21:08:45.492770 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:45.492778 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:45.492853 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:45.493868 (+ 1015us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:45.493978 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.495230 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.495937 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.496422 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.496515 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:45.498677 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:45.498682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:45.498682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:45.498694 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:45.500175 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:45.500558 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:45.500561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:45.500561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:45.500696 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:45.500926 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:45.500928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:45.500929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:45.501225 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:45.501382 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:45.501483 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 21:08:45.501622 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260812 21:08:45.509779 18618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b894eb7ba1864573ba5dd1282116493c: Generated new TSK 0
I20260812 21:08:45.509972 18618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:45.544623 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40481
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:45.628185 18648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:45.628423 18648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:45.628451 18648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:45.629675 18648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:45.629724 18648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:45.629750 18648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:45.631312 18648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:45.631546 18648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:45.631788 18648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:45.636786 18656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.636772 18654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.636772 18653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:45.637099 18648 server_base.cc:1034] running on GCE node
I20260812 21:08:45.637306 18648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:45.637552 18648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:45.638765 18648 hybrid_clock.cc:648] HybridClock initialized: now 1786568925638745 us; error 37 us; skew 500 ppm
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:45.641567 18648 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:45.642068 18648 webserver.cc:460] Webserver started at http://127.11.207.129:45483/ using document root <none> and password file <none>
I20260812 21:08:45.642300 18648 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:45.642349 18648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:45.642482 18648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:45.643530 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "f315a6181b22411b89dc4b5cfec4d0df"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "a928213abbe76c21b1f74aa2cf7e2c45"
server_key_iv: "b75f854c1e92838045b6a16c60390195"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.643874 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "f315a6181b22411b89dc4b5cfec4d0df"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "a928213abbe76c21b1f74aa2cf7e2c45"
server_key_iv: "b75f854c1e92838045b6a16c60390195"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.645249 18648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 21:08:45.645962 18666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.646107 18648 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:45.646196 18648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "f315a6181b22411b89dc4b5cfec4d0df"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "a928213abbe76c21b1f74aa2cf7e2c45"
server_key_iv: "b75f854c1e92838045b6a16c60390195"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.646260 18648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:45.675614 18648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:45.677734 18648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:45.677889 18648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:45.678135 18648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:45.678455 18648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:45.678490 18648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.678521 18648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:45.678540 18648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.684401 18648 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:34859
I20260812 21:08:45.684481 18872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:34859 every 8 connection(s)
I20260812 21:08:45.684788 18648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:45.690399 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18648
I20260812 21:08:45.690517 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:45.690739 12094 external_mini_cluster.cc:1428] Setting key 83020b1091cd460b9bdd6088e554066f
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:45.694379 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:45.685587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:39469 (local address 127.11.207.190:40481)
0812 21:08:45.685676 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:45.685678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:45.685828 (+ 150us) server_negotiation.cc:407] Connection header received
0812 21:08:45.686819 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:45.686821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:45.686834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:45.686883 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:45.687554 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.688065 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.689088 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.689294 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.689352 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:45.691629 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:45.691630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:45.691630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:45.691639 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:45.693072 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:45.693467 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:45.693468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:45.693468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:45.693528 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:45.693754 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:45.693755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:45.693756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:45.693874 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:45.693932 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:45.694202 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 21:08:45.694290 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 21:08:45.694548 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:45.685499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:40481 (local address 127.11.207.129:39469)
0812 21:08:45.685679 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:45.685731 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:45.686705 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:45.686968 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:45.686973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:45.687242 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:45.687449 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:45.687453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.688157 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.688158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:45.689002 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:45.689004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.689366 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.689367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:45.689498 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:45.689975 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:45.689984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:45.691535 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:45.693155 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:45.693163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:45.693166 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:45.693398 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:45.693581 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:45.693582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:45.693583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:45.693692 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:45.693929 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:45.693937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:45.694076 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:08:45.694167 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:45.694330 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260812 21:08:45.695088 18873 heartbeater.cc:347] Connected to a master server at 127.11.207.190:40481
I20260812 21:08:45.695189 18873 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:45.695369 18873 heartbeater.cc:511] Master 127.11.207.190:40481 requested a full tablet report, sending...
I20260812 21:08:45.695960 18526 ts_manager.cc:194] Registered new tserver with Master: f315a6181b22411b89dc4b5cfec4d0df (127.11.207.129:34859)
I20260812 21:08:45.696928 18526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:39469
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:45.718745 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40481
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:45.803454 18879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:45.803668 18879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:45.803689 18879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:45.804921 18879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:45.804963 18879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:45.804989 18879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:45.806526 18879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:45.806766 18879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:45.806986 18879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:08:45.812049 18879 server_base.cc:1034] running on GCE node
W20260812 21:08:45.812104 18892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.812021 18886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.812034 18887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:45.812453 18879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:45.812724 18879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:45.813900 18879 hybrid_clock.cc:648] HybridClock initialized: now 1786568925813885 us; error 27 us; skew 500 ppm
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:45.817278 18879 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:45.817787 18879 webserver.cc:460] Webserver started at http://127.11.207.130:33079/ using document root <none> and password file <none>
I20260812 21:08:45.818018 18879 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:45.818077 18879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:45.818184 18879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:45.819443 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "598dc3d957404c7b8068d2c35c9390e7"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "351bfce99b09f54c9a120ad6acd7d217"
server_key_iv: "025e30077433ebb45425ff8fe35e71c7"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.819856 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "598dc3d957404c7b8068d2c35c9390e7"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "351bfce99b09f54c9a120ad6acd7d217"
server_key_iv: "025e30077433ebb45425ff8fe35e71c7"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.821280 18879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:45.821976 18904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.822154 18879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.822247 18879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "598dc3d957404c7b8068d2c35c9390e7"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "351bfce99b09f54c9a120ad6acd7d217"
server_key_iv: "025e30077433ebb45425ff8fe35e71c7"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.822317 18879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:45.834865 18879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:45.837200 18879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:45.837318 18879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:45.837498 18879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:45.837814 18879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:45.837857 18879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.837880 18879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:45.837896 18879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.843699 18879 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:35215
I20260812 21:08:45.843743 19093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:35215 every 8 connection(s)
I20260812 21:08:45.844080 18879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:45.844630 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18879
I20260812 21:08:45.844735 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:45.844924 12094 external_mini_cluster.cc:1428] Setting key 1f31d6c3b123df66b03820fc86fdf83d
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
I20260812 21:08:45.855379 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:45.845510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:48983 (local address 127.11.207.190:40481)
0812 21:08:45.845572 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:45.845574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:45.845810 (+ 236us) server_negotiation.cc:407] Connection header received
0812 21:08:45.847007 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:45.847008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:45.847019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:45.847070 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:45.847912 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.848477 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.849858 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.850081 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.850161 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:45.852717 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:45.852719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:45.852719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:45.852733 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:45.854051 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:45.854440 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:45.854442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:45.854442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:45.854497 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:45.854752 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:45.854753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:45.854753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:45.854867 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:45.854925 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:45.855192 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 21:08:45.855285 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:08:45.855567 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:45.845493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:40481 (local address 127.11.207.130:48983)
0812 21:08:45.845673 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:45.845718 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:45.846892 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:45.847150 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:45.847154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:45.847520 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:45.847816 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:45.847823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.848558 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.848559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:45.849753 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:45.849757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:45.850181 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:45.850182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:45.850365 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:45.850972 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:45.850984 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:45.852619 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:45.854128 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:45.854136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:45.854140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:45.854370 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:45.854551 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:45.854552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:45.854552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:45.854689 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:45.854924 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:45.854932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:45.855074 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 21:08:45.855165 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:45.855345 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260812 21:08:45.856058 19096 heartbeater.cc:347] Connected to a master server at 127.11.207.190:40481
I20260812 21:08:45.856161 19096 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:45.856380 19096 heartbeater.cc:511] Master 127.11.207.190:40481 requested a full tablet report, sending...
I20260812 21:08:45.856865 18526 ts_manager.cc:194] Registered new tserver with Master: 598dc3d957404c7b8068d2c35c9390e7 (127.11.207.130:35215)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
I20260812 21:08:45.857299 18526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:48983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:45.871709 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40481
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:37441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:45.956056 19101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:45.956283 19101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:45.956319 19101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:45.957522 19101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:45.957566 19101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:45.957602 19101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:45.959122 19101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:45.959367 19101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:45.959580 19101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:45.964599 19114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.964612 19118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:45.964808 19113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:45.964798 19101 server_base.cc:1034] running on GCE node
I20260812 21:08:45.965034 19101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:45.965247 19101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:45.966372 19101 hybrid_clock.cc:648] HybridClock initialized: now 1786568925966364 us; error 28 us; skew 500 ppm
Aug 12 21:08:45 dist-test-slave-lkx2 krb5kdc[18447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:45.968904 19101 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:45.969357 19101 webserver.cc:460] Webserver started at http://127.11.207.131:43029/ using document root <none> and password file <none>
I20260812 21:08:45.969556 19101 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:45.969604 19101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:45.969734 19101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:45.970752 19101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "843ecc1cb31f40d8a4eb0b8839b8273c"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "1033140e8307e48652ec39429ef414ab"
server_key_iv: "5bd17f341d79d73f9e02b00a3ea5a9d3"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.971148 19101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "843ecc1cb31f40d8a4eb0b8839b8273c"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "1033140e8307e48652ec39429ef414ab"
server_key_iv: "5bd17f341d79d73f9e02b00a3ea5a9d3"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.972433 19101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:45.973093 19127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:45.973248 19101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:45.973333 19101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "843ecc1cb31f40d8a4eb0b8839b8273c"
format_stamp: "Formatted at 2026-08-12 21:08:45 on dist-test-slave-lkx2"
server_key: "1033140e8307e48652ec39429ef414ab"
server_key_iv: "5bd17f341d79d73f9e02b00a3ea5a9d3"
server_key_version: "encryptionkey@0"
I20260812 21:08:45.973402 19101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:46.001344 19101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:46.003539 19101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:46.003683 19101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:46.003931 19101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:46.004240 19101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:46.004276 19101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:46.004305 19101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:46.004330 19101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:46.010118 19101 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:34955
I20260812 21:08:46.010229 19337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:34955 every 8 connection(s)
I20260812 21:08:46.010509 19101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[18447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568925, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:46.018345 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19101
I20260812 21:08:46.018437 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:46.018606 12094 external_mini_cluster.cc:1428] Setting key 3a193e24a92dceac78c61368b4de3e81
I20260812 21:08:46.019646 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:46.011250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:44075 (local address 127.11.207.190:40481)
0812 21:08:46.011315 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:46.011317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:46.011481 (+ 164us) server_negotiation.cc:407] Connection header received
0812 21:08:46.012425 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:46.012427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:46.012442 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:46.012518 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:46.013142 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.013667 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.014618 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.014821 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.014894 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:46.017086 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:46.017087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:46.017088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:46.017097 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:46.018310 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:46.018717 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:46.018719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:46.018719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:46.018782 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:46.019025 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:46.019027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:46.019027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:46.019139 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:46.019197 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:46.019464 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 21:08:46.019558 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:08:46.019822 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:46.011201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:40481 (local address 127.11.207.131:44075)
0812 21:08:46.011356 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:46.011399 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:46.012300 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:46.012581 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:46.012584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:46.012844 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:46.013039 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:46.013043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.013749 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.013749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:46.014547 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:46.014550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.014964 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.014965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:46.015084 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:46.015583 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:46.015591 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:46.016985 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:46.018384 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:46.018390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:46.018393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:46.018626 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:46.018850 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:46.018851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:46.018851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:46.018967 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:46.019197 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:46.019204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:46.019345 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 21:08:46.019436 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:46.019596 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260812 21:08:46.020207 19338 heartbeater.cc:347] Connected to a master server at 127.11.207.190:40481
I20260812 21:08:46.020294 19338 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:46.020486 19338 heartbeater.cc:511] Master 127.11.207.190:40481 requested a full tablet report, sending...
I20260812 21:08:46.020809 18526 ts_manager.cc:194] Registered new tserver with Master: 843ecc1cb31f40d8a4eb0b8839b8273c (127.11.207.131:34955)
I20260812 21:08:46.021139 18526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:44075
I20260812 21:08:46.030320 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[18447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[18447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:46.054816 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:46.046935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36518 (local address 127.11.207.129:34859)
0812 21:08:46.047122 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:46.047127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:46.047138 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:46.047167 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:46.047169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:46.047202 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:46.047272 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:46.047804 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.048630 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.049435 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.049829 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.049893 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:46.052077 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:46.052086 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:46.052086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:46.052094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:46.053580 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:46.053983 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:46.053987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:46.053988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:46.054067 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:46.054300 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:46.054301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:46.054301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:46.054451 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:46.054559 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:46.054599 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 21:08:46.054666 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[18447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:46.062629 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:46.056180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36024 (local address 127.11.207.190:40481)
0812 21:08:46.056238 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:46.056240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:46.056248 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:46.056313 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:46.056313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:46.056323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:46.056387 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:46.056878 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.057388 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.058104 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:46.058299 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:46.058352 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:46.060529 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:46.060531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:46.060531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:46.060538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:46.061629 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:46.061945 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:46.061946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:46.061946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:46.061997 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:46.062207 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:46.062207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:46.062207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:46.062339 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:46.062408 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:46.062505 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 21:08:46.062563 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 21:08:46.062944 18526 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36024
I20260812 21:08:46.063316 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18648
I20260812 21:08:46.089390 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18879
I20260812 21:08:46.099388 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19101
I20260812 21:08:46.116326 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18469
2026-08-12T21:08:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2102 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[19362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[19362](info): set up 2 sockets
Aug 12 21:08:46 dist-test-slave-lkx2 krb5kdc[19362](info): commencing operation
krb5kdc: starting...
W20260812 21:08:47.328305 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:08:47.455670 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41543
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:47.540592 19387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:47.540831 19387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:47.540867 19387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:47.542042 19387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:47.542088 19387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:47.542110 19387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:47.542128 19387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:47.542148 19387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:47.543730 19387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:47.543992 19387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:47.544215 19387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:47.549242 19402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:47.549234 19403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:47.549237 19405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:47.549459 19387 server_base.cc:1034] running on GCE node
I20260812 21:08:47.549782 19387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:47.550066 19387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:47.551247 19387 hybrid_clock.cc:648] HybridClock initialized: now 1786568927551224 us; error 32 us; skew 500 ppm
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:47.558326 19387 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:47.558679 19387 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:47.558734 19387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:47.558880 19387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:47.559985 19387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "7d462f67c4ac43548f2571b468ec6036"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "701f4d2d0bf592e4a5e18f24a002919b"
server_key_iv: "3c9e19e3504ee5c6d2f4d92f0fd76bca"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.560336 19387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "7d462f67c4ac43548f2571b468ec6036"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "701f4d2d0bf592e4a5e18f24a002919b"
server_key_iv: "3c9e19e3504ee5c6d2f4d92f0fd76bca"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.561829 19387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:47.562647 19416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.562849 19387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:47.562956 19387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "7d462f67c4ac43548f2571b468ec6036"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "701f4d2d0bf592e4a5e18f24a002919b"
server_key_iv: "3c9e19e3504ee5c6d2f4d92f0fd76bca"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.563040 19387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:47.594455 19387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:47.596711 19387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:47.596879 19387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:47.601006 19387 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41543
I20260812 21:08:47.601086 19516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41543 every 8 connection(s)
I20260812 21:08:47.601367 19387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:47.602059 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19387
I20260812 21:08:47.602181 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:47.602371 12094 external_mini_cluster.cc:1428] Setting key 5a35670721dfb8ce8fcba50e8a28bbb1
I20260812 21:08:47.602555 19517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:47.605611 19517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036: Bootstrap starting.
I20260812 21:08:47.606246 19517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:47.606531 19517 log.cc:824] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:47.607345 19517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036: No bootstrap required, opened a new log
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:47.609460 19517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d462f67c4ac43548f2571b468ec6036" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41543 } }
I20260812 21:08:47.609647 19517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:47.609697 19517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d462f67c4ac43548f2571b468ec6036, State: Initialized, Role: FOLLOWER
I20260812 21:08:47.609840 19517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d462f67c4ac43548f2571b468ec6036" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41543 } }
I20260812 21:08:47.609916 19517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:47.609951 19517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:47.609999 19517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:47.610616 19517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d462f67c4ac43548f2571b468ec6036" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41543 } }
I20260812 21:08:47.610731 19517 leader_election.cc:302] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d462f67c4ac43548f2571b468ec6036; no voters:
I20260812 21:08:47.610906 19517 leader_election.cc:288] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:47.610962 19525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:47.611132 19525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [term 1 LEADER]: Becoming Leader. State: Replica: 7d462f67c4ac43548f2571b468ec6036, State: Running, Role: LEADER
I20260812 21:08:47.611195 19517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:47.611250 19525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d462f67c4ac43548f2571b468ec6036" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41543 } }
I20260812 21:08:47.611660 19527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d462f67c4ac43548f2571b468ec6036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d462f67c4ac43548f2571b468ec6036" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41543 } } }
I20260812 21:08:47.611657 19525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d462f67c4ac43548f2571b468ec6036. Latest consensus state: current_term: 1 leader_uuid: "7d462f67c4ac43548f2571b468ec6036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d462f67c4ac43548f2571b468ec6036" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41543 } } }
I20260812 21:08:47.611763 19527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:47.611780 19525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:47.611984 19538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:47.612586 19538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:47.613708 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:47.602956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.11.207.190:41543)
0812 21:08:47.603188 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:47.603196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:47.603224 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:08:47.603674 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:47.603682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:47.603745 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:47.603890 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:47.604608 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.605508 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.606356 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.606860 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.606988 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:47.609709 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:47.609714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:47.609714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:47.609729 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:47.611711 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:47.612155 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:47.612158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:47.612158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:47.612266 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:47.612514 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:47.612515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:47.612516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:47.612831 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:47.613398 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:47.613459 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:08:47.613561 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260812 21:08:47.614707 19538 catalog_manager.cc:1347] Generated new cluster ID: 1b3b4525a60b4db58e03324e7687efd8
I20260812 21:08:47.614751 19538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:47.628273 19538 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:47.628897 19538 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:47.639081 19538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036: Generated new TSK 0
I20260812 21:08:47.639310 19538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:47.671021 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41543
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:47.755765 19566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:47.755985 19566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:47.756009 19566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:47.757490 19566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:47.757547 19566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:47.757591 19566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:47.759142 19566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:47.759352 19566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:47.759541 19566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:47.764503 19579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:47.764516 19581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:47.764528 19585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:47.764782 19566 server_base.cc:1034] running on GCE node
I20260812 21:08:47.764953 19566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:47.765141 19566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:47.766261 19566 hybrid_clock.cc:648] HybridClock initialized: now 1786568927766248 us; error 29 us; skew 500 ppm
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:47.768749 19566 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:47.769081 19566 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:47.769130 19566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:47.769254 19566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:47.770306 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "b655f3dc7f7c40e18a2184849a05b1e4"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "d9455b9f7511aeb022285f178017d9b5"
server_key_iv: "6d97e0725d382f827db05f4749505754"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.770637 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "b655f3dc7f7c40e18a2184849a05b1e4"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "d9455b9f7511aeb022285f178017d9b5"
server_key_iv: "6d97e0725d382f827db05f4749505754"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.771874 19566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:08:47.772568 19594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.772702 19566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:47.772789 19566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "b655f3dc7f7c40e18a2184849a05b1e4"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "d9455b9f7511aeb022285f178017d9b5"
server_key_iv: "6d97e0725d382f827db05f4749505754"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.772851 19566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:47.802134 19566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:47.803946 19566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:47.804090 19566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:47.804380 19566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:47.804729 19566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:47.804764 19566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.804795 19566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:47.804812 19566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.810653 19566 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:37129
I20260812 21:08:47.810705 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:37129 every 8 connection(s)
I20260812 21:08:47.810964 19566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:47.817236 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19566
I20260812 21:08:47.817338 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:47.817504 12094 external_mini_cluster.cc:1428] Setting key f36f71b55f3b849a0802753daa3df39f
I20260812 21:08:47.820433 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:47.811704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:34755 (local address 127.11.207.190:41543)
0812 21:08:47.811835 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:47.811837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:47.812042 (+ 205us) server_negotiation.cc:407] Connection header received
0812 21:08:47.812770 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:47.812771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:47.812785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:47.812839 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:47.813920 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.814443 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.815460 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.815677 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.815741 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:47.817909 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:47.817911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:47.817912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:47.817920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:47.819151 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:47.819518 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:47.819520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:47.819520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:47.819574 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:47.819792 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:47.819793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:47.819794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:47.819900 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:47.819957 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:47.820217 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 21:08:47.820307 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 21:08:47.820600 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:47.811671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41543 (local address 127.11.207.129:34755)
0812 21:08:47.811897 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:47.811946 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:47.812630 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:47.812916 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:47.812921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:47.813202 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:47.813809 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:47.813816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.814532 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.814533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:47.815374 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:47.815376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.815750 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.815751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:47.815883 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:47.816373 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:47.816381 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:47.817821 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:47.819233 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:47.819240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:47.819242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:47.819454 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:47.819626 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:47.819627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:47.819628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:47.819737 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:47.819953 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:47.819961 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:47.820100 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:08:47.820189 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:08:47.820381 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260812 21:08:47.821244 19793 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41543
I20260812 21:08:47.821352 19793 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:47.821501 19793 heartbeater.cc:511] Master 127.11.207.190:41543 requested a full tablet report, sending...
I20260812 21:08:47.822003 19442 ts_manager.cc:194] Registered new tserver with Master: b655f3dc7f7c40e18a2184849a05b1e4 (127.11.207.129:37129)
I20260812 21:08:47.822615 19442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:34755
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:47.845230 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41543
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:47.929945 19802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:47.930152 19802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:47.930176 19802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:47.931358 19802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:47.931401 19802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:47.931427 19802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:47.932989 19802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:47.933214 19802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:47.933447 19802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:47.938410 19814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:47.938405 19811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:47.938524 19802 server_base.cc:1034] running on GCE node
W20260812 21:08:47.938405 19817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:47.938807 19802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:47.939046 19802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:47.940200 19802 hybrid_clock.cc:648] HybridClock initialized: now 1786568927940178 us; error 37 us; skew 500 ppm
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:47.942660 19802 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:47.942983 19802 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:47.943035 19802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:47.943158 19802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:47.944195 19802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "a33d255005d2445793cd6f6cf816bbd9"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "b0c96354b0fd81132cb1491d4f4acdde"
server_key_iv: "6dc23864cdc7ed8447a9ca3d3ae987fe"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.944583 19802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "a33d255005d2445793cd6f6cf816bbd9"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "b0c96354b0fd81132cb1491d4f4acdde"
server_key_iv: "6dc23864cdc7ed8447a9ca3d3ae987fe"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.945791 19802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:47.946475 19828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.946619 19802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:47.946702 19802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "a33d255005d2445793cd6f6cf816bbd9"
format_stamp: "Formatted at 2026-08-12 21:08:47 on dist-test-slave-lkx2"
server_key: "b0c96354b0fd81132cb1491d4f4acdde"
server_key_iv: "6dc23864cdc7ed8447a9ca3d3ae987fe"
server_key_version: "encryptionkey@0"
I20260812 21:08:47.946764 19802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:47.983566 19802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:47.985518 19802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:47.985666 19802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:47.985904 19802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:47.986244 19802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:47.986287 19802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.986318 19802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:47.986339 19802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:47.991895 19802 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:34785
I20260812 21:08:47.991961 20028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:34785 every 8 connection(s)
I20260812 21:08:47.992291 19802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:08:47 dist-test-slave-lkx2 krb5kdc[19362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568927, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:48.000779 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:47.993064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:42329 (local address 127.11.207.190:41543)
0812 21:08:47.993122 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:47.993124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:47.993338 (+ 214us) server_negotiation.cc:407] Connection header received
0812 21:08:47.993860 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:47.993861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:47.993872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:47.993923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:47.994831 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.995349 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.996292 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.996499 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.996549 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:47.998559 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:47.998560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:47.998561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:47.998567 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:47.999645 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:47.999976 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:47.999976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:47.999977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:48.000020 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:48.000207 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:48.000207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:48.000207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:48.000318 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.000420 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.000643 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.000707 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:48.000960 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:47.993033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41543 (local address 127.11.207.130:42329)
0812 21:08:47.993196 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:47.993249 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:47.993761 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:47.993988 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:47.993991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:47.994254 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:47.994757 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:47.994763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.995414 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.995415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:47.996224 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:47.996226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:47.996560 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:47.996560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:47.996661 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:47.997067 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:47.997075 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:47.998473 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:47.999709 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:47.999717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:47.999718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:47.999923 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:48.000074 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:48.000075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:48.000075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:48.000165 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:48.000412 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.000418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.000537 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.000622 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.000746 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260812 21:08:48.001451 20030 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41543
I20260812 21:08:48.001557 20030 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:48.001638 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19802
I20260812 21:08:48.001713 20030 heartbeater.cc:511] Master 127.11.207.190:41543 requested a full tablet report, sending...
I20260812 21:08:48.001731 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:48.001897 12094 external_mini_cluster.cc:1428] Setting key 9ae3497e9ad7ab39069b63376560e7f4
I20260812 21:08:48.002106 19442 ts_manager.cc:194] Registered new tserver with Master: a33d255005d2445793cd6f6cf816bbd9 (127.11.207.130:34785)
I20260812 21:08:48.002701 19442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:42329
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:48.029127 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41543
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:48.113652 20039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:48.113898 20039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:48.113935 20039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:48.115196 20039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:48.115242 20039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:48.115270 20039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:48.116835 20039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:48.117034 20039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:48.117234 20039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:48.122206 20048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.122201 20052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.122201 20059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:48.122440 20039 server_base.cc:1034] running on GCE node
I20260812 21:08:48.122674 20039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:48.122925 20039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:48.124094 20039 hybrid_clock.cc:648] HybridClock initialized: now 1786568928124074 us; error 35 us; skew 500 ppm
Aug 12 21:08:48 dist-test-slave-lkx2 krb5kdc[19362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568928, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:48.126787 20039 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:48.127143 20039 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:48.127207 20039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:48.127341 20039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:48.128496 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "87282926c1354a3d8a3ab1d147ff5c87"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "45a23b73da8050cbd3c332420e9090c8"
server_key_iv: "04ea1f7f7277c716ce1add985acf501f"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.128841 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "87282926c1354a3d8a3ab1d147ff5c87"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "45a23b73da8050cbd3c332420e9090c8"
server_key_iv: "04ea1f7f7277c716ce1add985acf501f"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.130097 20039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:48.130869 20068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.131058 20039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:48.131165 20039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "87282926c1354a3d8a3ab1d147ff5c87"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "45a23b73da8050cbd3c332420e9090c8"
server_key_iv: "04ea1f7f7277c716ce1add985acf501f"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.131227 20039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:48.146450 20039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:48.148723 20039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:48.148869 20039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:48.149118 20039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:48.149477 20039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:48.149515 20039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.149547 20039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:48.149571 20039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.155340 20039 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:36039
I20260812 21:08:48.155395 20262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:36039 every 8 connection(s)
I20260812 21:08:48.155656 20039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:08:48 dist-test-slave-lkx2 krb5kdc[19362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568928, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:48.164870 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.156397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:35069 (local address 127.11.207.190:41543)
0812 21:08:48.156446 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.156447 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.156665 (+ 218us) server_negotiation.cc:407] Connection header received
0812 21:08:48.157251 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.157253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.157267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.157317 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.158293 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.158797 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.159772 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.159968 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.160024 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.162230 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.162232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.162232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:48.162242 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.163455 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:48.163886 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:48.163887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:48.163887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:48.163951 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:48.164188 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:48.164189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:48.164189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:48.164310 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.164410 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.164678 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.164772 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:08:48.165057 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.156336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41543 (local address 127.11.207.131:35069)
0812 21:08:48.156542 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.156581 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.157139 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.157404 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.157409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.157683 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.158195 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.158201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.158881 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.158883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.159689 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.159691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.160038 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.160039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.160169 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.160689 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:48.160699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.162113 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.163539 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:48.163547 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:48.163549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:48.163808 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:48.164014 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:48.164015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:48.164015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:48.164122 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:48.164399 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.164406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.164548 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.164657 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.164817 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260812 21:08:48.165129 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20039
I20260812 21:08:48.165208 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:48.165386 12094 external_mini_cluster.cc:1428] Setting key 6f881159f0aa7ae1f9e9186824babae2
I20260812 21:08:48.165787 20263 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41543
I20260812 21:08:48.165897 20263 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:48.166066 20263 heartbeater.cc:511] Master 127.11.207.190:41543 requested a full tablet report, sending...
I20260812 21:08:48.166452 19442 ts_manager.cc:194] Registered new tserver with Master: 87282926c1354a3d8a3ab1d147ff5c87 (127.11.207.131:36039)
I20260812 21:08:48.166888 19442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:35069
I20260812 21:08:48.177110 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:08:48.184504 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.178401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.11.207.190:41543)
0812 21:08:48.178457 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.178459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.178555 (+ 96us) server_negotiation.cc:407] Connection header received
0812 21:08:48.178602 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.178603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.178612 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.178668 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.179194 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.179699 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.180517 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.180744 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.180804 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.182003 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.182004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.182004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:48.182011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.183337 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:48.183635 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:48.183636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:48.183636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:48.183717 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:48.183948 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:48.183950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:48.183950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:48.184119 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.184224 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.184291 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.184413 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:08:48.186023 19442 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:48.187655 19442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:48.193774 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.190052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:36039 (local address 127.0.0.1:51496)
0812 21:08:48.190554 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.190560 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.190593 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.190885 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.190886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.190889 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:48.190944 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.190946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.191895 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.191896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.192779 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.192781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.193475 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.193476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.193554 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.193570 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.193636 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.193683 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":115,"threads_started":1}
I20260812 21:08:48.193787 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.190304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.11.207.131:36039)
0812 21:08:48.190514 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.190519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.190597 (+ 78us) server_negotiation.cc:407] Connection header received
0812 21:08:48.190641 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.190644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.190659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.190791 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:48.191022 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.191799 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.192857 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.193399 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.193473 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.193531 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.193655 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.193695 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260812 21:08:48.194501 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.190042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37129 (local address 127.0.0.1:59622)
0812 21:08:48.190330 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.190375 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.190440 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.190760 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.190763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.190766 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:48.190840 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.190842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.192144 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.192145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.193244 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.193247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.194265 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.194266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.194331 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.194334 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.194392 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.194433 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260812 21:08:48.194533 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.190265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.11.207.129:37129)
0812 21:08:48.190599 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.190603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.190611 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:48.190644 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.190646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.190660 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.190742 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:48.190936 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.192026 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.193341 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.194168 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.194259 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.194312 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.194410 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.194456 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":86,"threads_started":1}
I20260812 21:08:48.194520 20161 tablet_service.cc:1467] Processing CreateTablet for tablet 9ac0e2f014ab42799269cbaa08d7f714 (DEFAULT_TABLE table=test-table [id=974d41b6d19449c6a89cd1e890d09724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:48.194923 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.190042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:34785 (local address 127.0.0.1:39574)
0812 21:08:48.190439 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.190449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.190500 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.191143 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.191144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.191146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:48.191185 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.191186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.192437 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.192438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.193765 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.193768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.194707 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.194708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.194770 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.194774 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.194821 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.194858 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":1,"thread_start_us":45,"threads_started":1}
I20260812 21:08:48.194936 20161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ac0e2f014ab42799269cbaa08d7f714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.195393 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.190277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.11.207.130:34785)
0812 21:08:48.190903 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.190907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.190916 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:48.190955 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.190956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.190971 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.191049 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:48.191280 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.192311 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.193852 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.194560 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.194620 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.194671 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.195066 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.195315 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":78,"threads_started":1}
I20260812 21:08:48.195393 19675 tablet_service.cc:1467] Processing CreateTablet for tablet 9ac0e2f014ab42799269cbaa08d7f714 (DEFAULT_TABLE table=test-table [id=974d41b6d19449c6a89cd1e890d09724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:48.195763 19675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ac0e2f014ab42799269cbaa08d7f714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.196041 19917 tablet_service.cc:1467] Processing CreateTablet for tablet 9ac0e2f014ab42799269cbaa08d7f714 (DEFAULT_TABLE table=test-table [id=974d41b6d19449c6a89cd1e890d09724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:48.196414 19917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ac0e2f014ab42799269cbaa08d7f714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.198370 20300 tablet_bootstrap.cc:492] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Bootstrap starting.
I20260812 21:08:48.198553 20302 tablet_bootstrap.cc:492] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4: Bootstrap starting.
I20260812 21:08:48.199028 20300 tablet_bootstrap.cc:654] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.199211 20302 tablet_bootstrap.cc:654] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.199594 20304 tablet_bootstrap.cc:492] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Bootstrap starting.
I20260812 21:08:48.199596 20300 log.cc:824] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.199793 20302 log.cc:824] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.200275 20304 tablet_bootstrap.cc:654] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.200371 20300 tablet_bootstrap.cc:492] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: No bootstrap required, opened a new log
I20260812 21:08:48.200443 20300 ts_tablet_manager.cc:1397] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:48.200624 20302 tablet_bootstrap.cc:492] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4: No bootstrap required, opened a new log
I20260812 21:08:48.200690 20302 ts_tablet_manager.cc:1397] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:48.200835 20304 log.cc:824] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.201637 20304 tablet_bootstrap.cc:492] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: No bootstrap required, opened a new log
I20260812 21:08:48.201705 20304 ts_tablet_manager.cc:1397] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:08:48.202356 20300 raft_consensus.cc:348] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.202531 20300 raft_consensus.cc:374] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.202571 20300 raft_consensus.cc:729] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87282926c1354a3d8a3ab1d147ff5c87, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.202575 20302 raft_consensus.cc:348] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.202714 20302 raft_consensus.cc:374] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.202759 20302 raft_consensus.cc:729] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b655f3dc7f7c40e18a2184849a05b1e4, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.202869 20300 consensus_queue.cc:260] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.202868 20302 consensus_queue.cc:260] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.203159 20302 ts_tablet_manager.cc:1428] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 21:08:48.203159 20300 ts_tablet_manager.cc:1428] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:08:48.203289 19793 heartbeater.cc:503] Master 127.11.207.190:41543 was elected leader, sending a full tablet report...
I20260812 21:08:48.203547 20263 heartbeater.cc:503] Master 127.11.207.190:41543 was elected leader, sending a full tablet report...
I20260812 21:08:48.203887 20304 raft_consensus.cc:348] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.204054 20304 raft_consensus.cc:374] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.204101 20304 raft_consensus.cc:729] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33d255005d2445793cd6f6cf816bbd9, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.204212 20304 consensus_queue.cc:260] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.204504 20030 heartbeater.cc:503] Master 127.11.207.190:41543 was elected leader, sending a full tablet report...
I20260812 21:08:48.204514 20304 ts_tablet_manager.cc:1428] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:08:48.207455 20311 raft_consensus.cc:482] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:48.207532 20311 raft_consensus.cc:504] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.207816 20311 leader_election.cc:288] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87282926c1354a3d8a3ab1d147ff5c87 (127.11.207.131:36039), b655f3dc7f7c40e18a2184849a05b1e4 (127.11.207.129:37129)
I20260812 21:08:48.211609 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.207977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37129 (local address 127.11.207.130:38319)
0812 21:08:48.208092 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.208104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.208161 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.208318 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.208319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.208320 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:48.208424 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.208426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.209329 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.209330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.210603 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.210606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.211366 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.211368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.211441 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.211445 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.211490 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.211542 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":5}
I20260812 21:08:48.211618 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.208038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:38319 (local address 127.11.207.129:37129)
0812 21:08:48.208087 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.208089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.208145 (+ 56us) server_negotiation.cc:407] Connection header received
0812 21:08:48.208216 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.208217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.208246 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.208304 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:48.208514 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.209247 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.210711 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.211298 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.211371 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.211380 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.211508 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.211553 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:48.211772 20314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.207983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:36039 (local address 127.11.207.130:39133)
0812 21:08:48.208161 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.208170 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.208235 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.208380 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.208380 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.208382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:48.208558 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.208560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.209382 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.209382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.210603 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:48.210606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.211439 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.211440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:48.211602 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.211606 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.211660 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.211699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":49,"thread_start_us":91,"threads_started":1}
I20260812 21:08:48.211797 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.208057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:39133 (local address 127.11.207.131:36039)
0812 21:08:48.208096 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.208098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.208237 (+ 139us) server_negotiation.cc:407] Connection header received
0812 21:08:48.208282 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.208282 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.208290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.208361 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:48.208622 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.209320 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.210710 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.211335 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.211372 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.211380 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.211693 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.211738 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:08:48.211972 19710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac0e2f014ab42799269cbaa08d7f714" candidate_uuid: "a33d255005d2445793cd6f6cf816bbd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" is_pre_election: true
I20260812 21:08:48.212054 20193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac0e2f014ab42799269cbaa08d7f714" candidate_uuid: "a33d255005d2445793cd6f6cf816bbd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87282926c1354a3d8a3ab1d147ff5c87" is_pre_election: true
I20260812 21:08:48.212152 19710 raft_consensus.cc:2454] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33d255005d2445793cd6f6cf816bbd9 in term 0.
I20260812 21:08:48.212193 20193 raft_consensus.cc:2454] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a33d255005d2445793cd6f6cf816bbd9 in term 0.
I20260812 21:08:48.212412 19834 leader_election.cc:302] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87282926c1354a3d8a3ab1d147ff5c87, a33d255005d2445793cd6f6cf816bbd9; no voters:
I20260812 21:08:48.212527 20311 raft_consensus.cc:2781] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:48.212587 20311 raft_consensus.cc:482] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:48.212602 20311 raft_consensus.cc:3037] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.213269 20311 raft_consensus.cc:504] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.213403 20311 leader_election.cc:288] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [CANDIDATE]: Term 1 election: Requested vote from peers 87282926c1354a3d8a3ab1d147ff5c87 (127.11.207.131:36039), b655f3dc7f7c40e18a2184849a05b1e4 (127.11.207.129:37129)
I20260812 21:08:48.213601 20193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac0e2f014ab42799269cbaa08d7f714" candidate_uuid: "a33d255005d2445793cd6f6cf816bbd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87282926c1354a3d8a3ab1d147ff5c87"
I20260812 21:08:48.213672 20193 raft_consensus.cc:3037] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.214565 20193 raft_consensus.cc:2454] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33d255005d2445793cd6f6cf816bbd9 in term 1.
I20260812 21:08:48.214671 19710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac0e2f014ab42799269cbaa08d7f714" candidate_uuid: "a33d255005d2445793cd6f6cf816bbd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b655f3dc7f7c40e18a2184849a05b1e4"
I20260812 21:08:48.214725 19834 leader_election.cc:302] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87282926c1354a3d8a3ab1d147ff5c87, a33d255005d2445793cd6f6cf816bbd9; no voters:
I20260812 21:08:48.214753 19710 raft_consensus.cc:3037] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.214805 20311 raft_consensus.cc:2781] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:48.214936 20311 raft_consensus.cc:686] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 1 LEADER]: Becoming Leader. State: Replica: a33d255005d2445793cd6f6cf816bbd9, State: Running, Role: LEADER
I20260812 21:08:48.215019 20311 consensus_queue.cc:237] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } }
I20260812 21:08:48.215679 19710 raft_consensus.cc:2454] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a33d255005d2445793cd6f6cf816bbd9 in term 1.
I20260812 21:08:48.215827 19440 catalog_manager.cc:5495] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a33d255005d2445793cd6f6cf816bbd9 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "a33d255005d2445793cd6f6cf816bbd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a33d255005d2445793cd6f6cf816bbd9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 } health_report { overall_health: UNKNOWN } } }
I20260812 21:08:48.220762 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.218614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.11.207.130:34785)
0812 21:08:48.218659 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.218661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.218670 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:48.218717 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.218717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.218731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.218781 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:48.218920 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.219378 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.220091 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:48.220281 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:48.220330 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:48.220500 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:48.220597 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:48.220656 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.220673 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.220691 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:48.222373 20193 raft_consensus.cc:1261] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 1 FOLLOWER]: Refusing update from remote peer a33d255005d2445793cd6f6cf816bbd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:48.222396 19710 raft_consensus.cc:1261] T 9ac0e2f014ab42799269cbaa08d7f714 P b655f3dc7f7c40e18a2184849a05b1e4 [term 1 FOLLOWER]: Refusing update from remote peer a33d255005d2445793cd6f6cf816bbd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:48.222715 20311 consensus_queue.cc:1035] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87282926c1354a3d8a3ab1d147ff5c87" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:48.222860 20315 consensus_queue.cc:1035] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b655f3dc7f7c40e18a2184849a05b1e4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:48.227787 19440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:48.227892 19440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:48.228667 19440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d462f67c4ac43548f2571b468ec6036: Sending DeleteTablet for 3 replicas of tablet 9ac0e2f014ab42799269cbaa08d7f714
I20260812 21:08:48.229002 20161 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ac0e2f014ab42799269cbaa08d7f714 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:48 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51496
I20260812 21:08:48.229024 19675 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ac0e2f014ab42799269cbaa08d7f714 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:48 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:59622
I20260812 21:08:48.229211 20331 tablet_replica.cc:330] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: stopping tablet replica
I20260812 21:08:48.229321 20331 raft_consensus.cc:2229] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:48.229360 19917 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ac0e2f014ab42799269cbaa08d7f714 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:48 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:39574
I20260812 21:08:48.229417 20331 raft_consensus.cc:2258] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:48.229624 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19566
I20260812 21:08:48.229645 20333 tablet_replica.cc:330] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: stopping tablet replica
I20260812 21:08:48.245409 20333 raft_consensus.cc:2229] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:48.245592 20333 raft_consensus.cc:2258] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:48.246312 20333 ts_tablet_manager.cc:1905] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:48.246729 20331 ts_tablet_manager.cc:1905] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:48.248099 20333 ts_tablet_manager.cc:1918] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:48.248188 20333 log.cc:1196] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/9ac0e2f014ab42799269cbaa08d7f714
I20260812 21:08:48.248503 20333 ts_tablet_manager.cc:1939] T 9ac0e2f014ab42799269cbaa08d7f714 P a33d255005d2445793cd6f6cf816bbd9: Deleting consensus metadata
I20260812 21:08:48.248677 20331 ts_tablet_manager.cc:1918] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:48.248768 20331 log.cc:1196] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/9ac0e2f014ab42799269cbaa08d7f714
I20260812 21:08:48.248998 20331 ts_tablet_manager.cc:1939] T 9ac0e2f014ab42799269cbaa08d7f714 P 87282926c1354a3d8a3ab1d147ff5c87: Deleting consensus metadata
I20260812 21:08:48.249019 19419 catalog_manager.cc:4841] TS a33d255005d2445793cd6f6cf816bbd9 (127.11.207.130:34785): tablet 9ac0e2f014ab42799269cbaa08d7f714 (table test-table [id=974d41b6d19449c6a89cd1e890d09724]) successfully deleted
I20260812 21:08:48.249380 19421 catalog_manager.cc:4841] TS 87282926c1354a3d8a3ab1d147ff5c87 (127.11.207.131:36039): tablet 9ac0e2f014ab42799269cbaa08d7f714 (table test-table [id=974d41b6d19449c6a89cd1e890d09724]) successfully deleted
W20260812 21:08:48.253674 19418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:37129 (error 108)
I20260812 21:08:48.254282 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.254151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37129 (local address 127.0.0.1:59630)
0812 21:08:48.254207 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.254250 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 21:08:48.254285 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19802
W20260812 21:08:48.264782 19418 catalog_manager.cc:4584] TS b655f3dc7f7c40e18a2184849a05b1e4 (127.11.207.129:37129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ac0e2f014ab42799269cbaa08d7f714: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37129: connect: Connection refused (error 111)
I20260812 21:08:48.269111 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20039
I20260812 21:08:48.280658 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19387
2026-08-12T21:08:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2168 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T21:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:48Z Disabled control of system clock
I20260812 21:08:48.316262 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34833
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:08:48.398011 20341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:48.398227 20341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:48.398265 20341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:48.399434 20341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:48.399477 20341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:48.399494 20341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:48.399510 20341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:48.399525 20341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:48.400918 20341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:48.401156 20341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:48.401448 20341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:48.406239 20350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.406280 20354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.406304 20349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:48.406625 20341 server_base.cc:1034] running on GCE node
I20260812 21:08:48.406889 20341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:48.407166 20341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:48.408314 20341 hybrid_clock.cc:648] HybridClock initialized: now 1786568928408297 us; error 39 us; skew 500 ppm
I20260812 21:08:48.409552 20341 webserver.cc:460] Webserver started at http://127.11.207.190:40031/ using document root <none> and password file <none>
I20260812 21:08:48.409821 20341 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:48.409874 20341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:48.410008 20341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:48.411092 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "f52e2fa4b2be44669c77ac5b030fbb84"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "05776650e744b1bd15ca34c7b1c1c306"
server_key_iv: "6d981e6d24eccaef86924a92255af84d"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.411430 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "f52e2fa4b2be44669c77ac5b030fbb84"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "05776650e744b1bd15ca34c7b1c1c306"
server_key_iv: "6d981e6d24eccaef86924a92255af84d"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.412688 20341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:48.413443 20364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.413617 20341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:48.413704 20341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "f52e2fa4b2be44669c77ac5b030fbb84"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "05776650e744b1bd15ca34c7b1c1c306"
server_key_iv: "6d981e6d24eccaef86924a92255af84d"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.413765 20341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:48.417218 20341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:48.417480 20341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:48.417605 20341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:48.421473 20341 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:34833
I20260812 21:08:48.421509 20457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:34833 every 8 connection(s)
I20260812 21:08:48.421852 20341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:48.422508 20462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.425479 20462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84: Bootstrap starting.
I20260812 21:08:48.426018 20462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.426271 20462 log.cc:824] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.426911 20462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84: No bootstrap required, opened a new log
I20260812 21:08:48.428735 20462 raft_consensus.cc:348] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34833 } }
I20260812 21:08:48.428879 20462 raft_consensus.cc:374] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.428900 20462 raft_consensus.cc:729] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52e2fa4b2be44669c77ac5b030fbb84, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.429005 20462 consensus_queue.cc:260] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34833 } }
I20260812 21:08:48.429059 20462 raft_consensus.cc:388] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:48.429086 20462 raft_consensus.cc:482] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:48.429124 20462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.429663 20462 raft_consensus.cc:504] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34833 } }
I20260812 21:08:48.429803 20462 leader_election.cc:302] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f52e2fa4b2be44669c77ac5b030fbb84; no voters:
I20260812 21:08:48.429955 20462 leader_election.cc:288] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:48.430027 20467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:48.430190 20467 raft_consensus.cc:686] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [term 1 LEADER]: Becoming Leader. State: Replica: f52e2fa4b2be44669c77ac5b030fbb84, State: Running, Role: LEADER
I20260812 21:08:48.430202 20462 sys_catalog.cc:564] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:48.430298 20467 consensus_queue.cc:237] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34833 } }
I20260812 21:08:48.430642 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20341
I20260812 21:08:48.430761 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:48.430955 12094 external_mini_cluster.cc:1428] Setting key 2f5d4c7acd6e9b973fe01eed9bebe92c
I20260812 21:08:48.431421 20469 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f52e2fa4b2be44669c77ac5b030fbb84. Latest consensus state: current_term: 1 leader_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34833 } } }
I20260812 21:08:48.431507 20469 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:48.431596 20468 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e2fa4b2be44669c77ac5b030fbb84" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34833 } } }
I20260812 21:08:48.431674 20468 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:48.431701 20495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:48.432165 20495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:48.433830 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.431495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.11.207.190:34833)
0812 21:08:48.431869 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.431880 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.432083 (+ 203us) server_negotiation.cc:407] Connection header received
0812 21:08:48.432190 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.432195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.432236 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.432389 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.433018 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.433022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.433023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.433027 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.433068 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.433079 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.433142 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.433584 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.433647 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":87,"threads_started":1}
I20260812 21:08:48.434129 20495 catalog_manager.cc:1347] Generated new cluster ID: 63bf9b1336ed4226aa07cbbdb7892f08
I20260812 21:08:48.434186 20495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:48.440888 20495 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:48.441063 20495 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:48.448460 20495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84: Generated new TSK 0
I20260812 21:08:48.448645 20495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:48.452171 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34833
--builtin_ntp_servers=127.11.207.148:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:08:48.537164 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:48.537427 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:48.537469 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:48.538671 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:48.538715 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:48.538754 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:48.540135 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:48.540416 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:48.540747 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:48.546713 20521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.546780 20520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:48.546867 20506 server_base.cc:1034] running on GCE node
W20260812 21:08:48.546803 20524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:48.547223 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:48.547509 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:48.548673 20506 hybrid_clock.cc:648] HybridClock initialized: now 1786568928548661 us; error 29 us; skew 500 ppm
I20260812 21:08:48.549985 20506 webserver.cc:460] Webserver started at http://127.11.207.129:44327/ using document root <none> and password file <none>
I20260812 21:08:48.550247 20506 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:48.550295 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:48.550412 20506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:48.551499 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "45306934f4024817ba39de431adf4fa8"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "0e3a6947318b6563dbc6b5fb653c1916"
server_key_iv: "1abee8387868b65ab59554a8325d813c"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.551822 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "45306934f4024817ba39de431adf4fa8"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "0e3a6947318b6563dbc6b5fb653c1916"
server_key_iv: "1abee8387868b65ab59554a8325d813c"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.553181 20506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:48.554001 20531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.554178 20506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:48.554270 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "45306934f4024817ba39de431adf4fa8"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "0e3a6947318b6563dbc6b5fb653c1916"
server_key_iv: "1abee8387868b65ab59554a8325d813c"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.554337 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:48.557646 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:48.557901 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:48.558020 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:48.558218 20506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:48.558507 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:48.558540 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.558566 20506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:48.558590 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.564594 20506 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:37735
I20260812 21:08:48.564666 20741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:37735 every 8 connection(s)
I20260812 21:08:48.564965 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:48.567863 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20506
I20260812 21:08:48.567960 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:48.568019 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.565653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:35815 (local address 127.11.207.190:34833)
0812 21:08:48.565738 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.565740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.565955 (+ 215us) server_negotiation.cc:407] Connection header received
0812 21:08:48.566405 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.566406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.566413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.566486 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.567766 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.567767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.567767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.567772 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.567793 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.567797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.567869 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.567931 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.567948 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 21:08:48.568167 12094 external_mini_cluster.cc:1428] Setting key 2410436d1ba14f49f1ec9fd14f16333c
I20260812 21:08:48.568287 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.565597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34833 (local address 127.11.207.129:35815)
0812 21:08:48.565810 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.565851 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.566310 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.566551 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.566554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.566563 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.567571 (+ 1008us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.567571 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.567600 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.567631 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.567868 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.567870 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.567872 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.567926 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.567994 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260812 21:08:48.568753 20743 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34833
I20260812 21:08:48.568849 20743 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:48.568996 20743 heartbeater.cc:511] Master 127.11.207.190:34833 requested a full tablet report, sending...
I20260812 21:08:48.569141 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34833
--builtin_ntp_servers=127.11.207.148:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 21:08:48.569533 20394 ts_manager.cc:194] Registered new tserver with Master: 45306934f4024817ba39de431adf4fa8 (127.11.207.129:37735)
W20260812 21:08:48.652623 20749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:48.652854 20749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:48.652887 20749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:48.654068 20749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:48.654117 20749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:48.654162 20749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:48.655536 20749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:48.655805 20749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:48.656102 20749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:48.661324 20758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.661356 20754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.661365 20755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:48.661404 20749 server_base.cc:1034] running on GCE node
I20260812 21:08:48.661701 20749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:48.661937 20749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:48.663084 20749 hybrid_clock.cc:648] HybridClock initialized: now 1786568928663071 us; error 31 us; skew 500 ppm
I20260812 21:08:48.664578 20749 webserver.cc:460] Webserver started at http://127.11.207.130:39627/ using document root <none> and password file <none>
I20260812 21:08:48.664826 20749 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:48.664880 20749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:48.665007 20749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:48.666069 20749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "6f6fd262a4504405b0b9fce9117ad93a"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "c8146dc8f0a282fdaca389bf846e785e"
server_key_iv: "ae1907057eaea6101d9558da12395c3d"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.666405 20749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "6f6fd262a4504405b0b9fce9117ad93a"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "c8146dc8f0a282fdaca389bf846e785e"
server_key_iv: "ae1907057eaea6101d9558da12395c3d"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.667658 20749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:48.668426 20766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.668591 20749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:48.668678 20749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "6f6fd262a4504405b0b9fce9117ad93a"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "c8146dc8f0a282fdaca389bf846e785e"
server_key_iv: "ae1907057eaea6101d9558da12395c3d"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.668751 20749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:48.671969 20749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:48.672211 20749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:48.672369 20749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:48.672566 20749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:48.672868 20749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:48.672900 20749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.672927 20749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:48.672951 20749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.678406 20749 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:39267
I20260812 21:08:48.678474 20970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:39267 every 8 connection(s)
I20260812 21:08:48.678772 20749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:48.681677 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.679479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:44395 (local address 127.11.207.190:34833)
0812 21:08:48.679558 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.679560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.679745 (+ 185us) server_negotiation.cc:407] Connection header received
0812 21:08:48.680244 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.680245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.680252 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.680306 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.681430 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.681431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.681432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.681437 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.681456 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.681460 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.681514 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.681590 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.681606 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 21:08:48.681900 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.679441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34833 (local address 127.11.207.130:44395)
0812 21:08:48.679621 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.679656 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.680127 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.680399 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.680403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.680412 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.681256 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.681256 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.681289 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.681319 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.681516 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.681522 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.681525 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.681597 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.681661 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260812 21:08:48.682322 20971 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34833
I20260812 21:08:48.682408 20971 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:48.682557 20971 heartbeater.cc:511] Master 127.11.207.190:34833 requested a full tablet report, sending...
I20260812 21:08:48.682888 20394 ts_manager.cc:194] Registered new tserver with Master: 6f6fd262a4504405b0b9fce9117ad93a (127.11.207.130:39267)
I20260812 21:08:48.684523 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20749
I20260812 21:08:48.684605 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:48.684747 12094 external_mini_cluster.cc:1428] Setting key e23e47e2da88a8d78689a395ae445274
I20260812 21:08:48.685550 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34833
--builtin_ntp_servers=127.11.207.148:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:08:48.768307 20976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:48.768525 20976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:48.768548 20976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:48.769647 20976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:48.769680 20976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:48.769703 20976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:48.770982 20976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:48.771188 20976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:48.771469 20976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:48.776643 20986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.776649 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:48.776700 20990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:48.776914 20976 server_base.cc:1034] running on GCE node
I20260812 21:08:48.777137 20976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:48.777374 20976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:48.778504 20976 hybrid_clock.cc:648] HybridClock initialized: now 1786568928778488 us; error 34 us; skew 500 ppm
I20260812 21:08:48.779796 20976 webserver.cc:460] Webserver started at http://127.11.207.131:40697/ using document root <none> and password file <none>
I20260812 21:08:48.780030 20976 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:48.780076 20976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:48.780202 20976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:48.781292 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "1f7b5aa69581470993e26ac53e4e2780"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "d140caaefe7e8b113a619f84b1d682f5"
server_key_iv: "c88d8f6e224328655998ba2949e17c81"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.781615 20976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "1f7b5aa69581470993e26ac53e4e2780"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "d140caaefe7e8b113a619f84b1d682f5"
server_key_iv: "c88d8f6e224328655998ba2949e17c81"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.782837 20976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:48.783468 20997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.783604 20976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:48.783690 20976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "1f7b5aa69581470993e26ac53e4e2780"
format_stamp: "Formatted at 2026-08-12 21:08:48 on dist-test-slave-lkx2"
server_key: "d140caaefe7e8b113a619f84b1d682f5"
server_key_iv: "c88d8f6e224328655998ba2949e17c81"
server_key_version: "encryptionkey@0"
I20260812 21:08:48.783764 20976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:48.787041 20976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:48.787293 20976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:48.787410 20976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:48.787595 20976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:48.787892 20976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:48.787925 20976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.787951 20976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:48.787974 20976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:48.793465 20976 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:44939
I20260812 21:08:48.793506 21192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:44939 every 8 connection(s)
I20260812 21:08:48.793808 20976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:48.796626 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.794519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:59347 (local address 127.11.207.190:34833)
0812 21:08:48.794595 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.794597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.794783 (+ 186us) server_negotiation.cc:407] Connection header received
0812 21:08:48.795254 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.795255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.795262 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.795323 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.796430 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.796431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.796431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.796435 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.796452 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.796455 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.796503 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.796557 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.796570 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:08:48.796803 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.794488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34833 (local address 127.11.207.131:59347)
0812 21:08:48.794661 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.794695 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.795145 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.795384 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.795387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.795396 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.796268 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.796269 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.796298 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.796326 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.796501 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.796504 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.796506 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.796558 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.796608 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260812 21:08:48.797287 21194 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34833
I20260812 21:08:48.797369 21194 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:48.797508 21194 heartbeater.cc:511] Master 127.11.207.190:34833 requested a full tablet report, sending...
I20260812 21:08:48.797868 20394 ts_manager.cc:194] Registered new tserver with Master: 1f7b5aa69581470993e26ac53e4e2780 (127.11.207.131:44939)
I20260812 21:08:48.800899 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20976
I20260812 21:08:48.800987 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:48.801158 12094 external_mini_cluster.cc:1428] Setting key fb6ae084d454a13b104bb5ae9bfca8df
I20260812 21:08:48.802002 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:08:48.804204 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.803317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.11.207.190:34833)
0812 21:08:48.803373 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.803374 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.803484 (+ 110us) server_negotiation.cc:407] Connection header received
0812 21:08:48.803540 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.803541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.803547 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.803582 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.804054 (+ 472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.804055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.804055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.804059 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.804072 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.804076 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.804116 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.804145 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.804155 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:08:48.805814 20394 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:48.807385 20394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:48.811408 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.809596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:39267 (local address 127.0.0.1:57962)
0812 21:08:48.810105 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.810113 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.810154 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.810604 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.810605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.810607 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.810990 (+ 383us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.810990 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.810997 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.811012 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.811240 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.811247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.811249 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.811319 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.811333 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":110,"threads_started":1}
I20260812 21:08:48.811425 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.809867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.11.207.130:39267)
0812 21:08:48.810058 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.810063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.810158 (+ 95us) server_negotiation.cc:407] Connection header received
0812 21:08:48.810210 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.810212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.810222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.810296 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.811074 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.811082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.811082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.811086 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.811125 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.811157 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.811213 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.811331 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.811346 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260812 21:08:48.811502 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.809596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37735 (local address 127.0.0.1:49624)
0812 21:08:48.810231 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.810239 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.810303 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.810731 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.810733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.810735 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.811129 (+ 394us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.811130 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.811135 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.811146 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.811370 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.811371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.811371 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.811418 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.811433 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":13,"thread_start_us":135,"threads_started":1}
I20260812 21:08:48.811520 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.809884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.11.207.129:37735)
0812 21:08:48.810135 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.810139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.810361 (+ 222us) server_negotiation.cc:407] Connection header received
0812 21:08:48.810393 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.810396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.810404 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.810465 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.811228 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.811237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.811238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.811242 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.811286 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.811317 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.811372 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.811424 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.811440 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
I20260812 21:08:48.812078 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.809585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44939 (local address 127.0.0.1:52420)
0812 21:08:48.809901 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.809944 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.810004 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.810313 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.810320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.810327 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.810831 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.810831 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.810847 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.810871 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.811943 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.811944 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.811944 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.811997 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.812011 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":60,"threads_started":1}
I20260812 21:08:48.812125 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.809690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.11.207.131:44939)
0812 21:08:48.810101 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.810105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.810117 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:08:48.810144 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.810146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.810155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.810223 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.811179 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.811183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.811183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.811187 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.811218 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.811232 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.811311 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.812031 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.812048 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":88,"threads_started":1}
I20260812 21:08:48.812318 20613 tablet_service.cc:1467] Processing CreateTablet for tablet b622b884f3ff461b93812bbcd8510ce5 (DEFAULT_TABLE table=test-table [id=2ac768992be6415c9a91fc4b2724a8f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:48.812493 20864 tablet_service.cc:1467] Processing CreateTablet for tablet b622b884f3ff461b93812bbcd8510ce5 (DEFAULT_TABLE table=test-table [id=2ac768992be6415c9a91fc4b2724a8f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:48.812796 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b622b884f3ff461b93812bbcd8510ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.812796 20613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b622b884f3ff461b93812bbcd8510ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.816252 21226 tablet_bootstrap.cc:492] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8: Bootstrap starting.
I20260812 21:08:48.816434 21225 tablet_bootstrap.cc:492] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Bootstrap starting.
I20260812 21:08:48.816844 21084 tablet_service.cc:1467] Processing CreateTablet for tablet b622b884f3ff461b93812bbcd8510ce5 (DEFAULT_TABLE table=test-table [id=2ac768992be6415c9a91fc4b2724a8f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:48.817109 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b622b884f3ff461b93812bbcd8510ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:48.817111 21225 tablet_bootstrap.cc:654] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.817111 21226 tablet_bootstrap.cc:654] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.817431 21225 log.cc:824] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.817430 21226 log.cc:824] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.818289 21225 tablet_bootstrap.cc:492] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: No bootstrap required, opened a new log
I20260812 21:08:48.818351 21225 ts_tablet_manager.cc:1397] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:48.818418 21226 tablet_bootstrap.cc:492] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8: No bootstrap required, opened a new log
I20260812 21:08:48.818468 21226 ts_tablet_manager.cc:1397] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:48.819723 21230 tablet_bootstrap.cc:492] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Bootstrap starting.
I20260812 21:08:48.820322 21230 tablet_bootstrap.cc:654] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:48.820632 21230 log.cc:824] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:48.821334 21230 tablet_bootstrap.cc:492] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: No bootstrap required, opened a new log
I20260812 21:08:48.821197 21226 raft_consensus.cc:348] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.821372 21226 raft_consensus.cc:374] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.821394 21230 ts_tablet_manager.cc:1397] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:48.821424 21226 raft_consensus.cc:729] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45306934f4024817ba39de431adf4fa8, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.821565 21226 consensus_queue.cc:260] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.821125 21225 raft_consensus.cc:348] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.821656 21225 raft_consensus.cc:374] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.821699 21225 raft_consensus.cc:729] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6fd262a4504405b0b9fce9117ad93a, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.821864 21226 ts_tablet_manager.cc:1428] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 21:08:48.821828 21225 consensus_queue.cc:260] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.822019 20743 heartbeater.cc:503] Master 127.11.207.190:34833 was elected leader, sending a full tablet report...
I20260812 21:08:48.822088 21225 ts_tablet_manager.cc:1428] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 21:08:48.822305 20971 heartbeater.cc:503] Master 127.11.207.190:34833 was elected leader, sending a full tablet report...
I20260812 21:08:48.823587 21230 raft_consensus.cc:348] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.823704 21230 raft_consensus.cc:374] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:48.823734 21230 raft_consensus.cc:729] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7b5aa69581470993e26ac53e4e2780, State: Initialized, Role: FOLLOWER
I20260812 21:08:48.823843 21230 consensus_queue.cc:260] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.824059 21230 ts_tablet_manager.cc:1428] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:08:48.824106 21194 heartbeater.cc:503] Master 127.11.207.190:34833 was elected leader, sending a full tablet report...
I20260812 21:08:48.897393 21235 raft_consensus.cc:482] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:48.897526 21235 raft_consensus.cc:504] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.897861 21235 leader_election.cc:288] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f7b5aa69581470993e26ac53e4e2780 (127.11.207.131:44939), 45306934f4024817ba39de431adf4fa8 (127.11.207.129:37735)
I20260812 21:08:48.899061 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.898053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37735 (local address 127.11.207.130:47467)
0812 21:08:48.898129 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.898143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.898189 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.898343 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.898344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.898347 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.898738 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.898738 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.898747 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.898759 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.898944 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.898944 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.898945 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.898986 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.899000 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 21:08:48.899082 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.898155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:47467 (local address 127.11.207.129:37735)
0812 21:08:48.898213 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.898215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.898223 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:48.898259 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.898260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.898266 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.898327 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.898850 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.898852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.898852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.898857 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.898878 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.898883 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.898940 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.898997 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.899011 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:08:48.899173 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.898153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:54865 (local address 127.11.207.131:44939)
0812 21:08:48.898208 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.898210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.898342 (+ 132us) server_negotiation.cc:407] Connection header received
0812 21:08:48.898373 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.898374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.898379 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.898422 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.898973 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.898975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.898975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.898980 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.898996 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.899001 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.899055 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.899098 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.899110 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:08:48.899175 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.898055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44939 (local address 127.11.207.130:54865)
0812 21:08:48.898273 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.898281 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:48.898329 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:48.898438 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:48.898438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:48.898440 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:48.898879 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:08:48.898880 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:48.898884 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:08:48.898892 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:48.899055 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:48.899055 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:48.899056 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:08:48.899100 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:08:48.899110 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":21,"thread_start_us":124,"threads_started":1}
I20260812 21:08:48.899469 21114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b622b884f3ff461b93812bbcd8510ce5" candidate_uuid: "6f6fd262a4504405b0b9fce9117ad93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7b5aa69581470993e26ac53e4e2780" is_pre_election: true
I20260812 21:08:48.899482 20614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b622b884f3ff461b93812bbcd8510ce5" candidate_uuid: "6f6fd262a4504405b0b9fce9117ad93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45306934f4024817ba39de431adf4fa8" is_pre_election: true
I20260812 21:08:48.899614 20614 raft_consensus.cc:2454] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f6fd262a4504405b0b9fce9117ad93a in term 0.
I20260812 21:08:48.899614 21114 raft_consensus.cc:2454] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f6fd262a4504405b0b9fce9117ad93a in term 0.
I20260812 21:08:48.899853 20771 leader_election.cc:302] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45306934f4024817ba39de431adf4fa8, 6f6fd262a4504405b0b9fce9117ad93a; no voters:
I20260812 21:08:48.900010 21235 raft_consensus.cc:2781] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:48.900067 21235 raft_consensus.cc:482] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:48.900094 21235 raft_consensus.cc:3037] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.900874 21235 raft_consensus.cc:504] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.900995 21235 leader_election.cc:288] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [CANDIDATE]: Term 1 election: Requested vote from peers 1f7b5aa69581470993e26ac53e4e2780 (127.11.207.131:44939), 45306934f4024817ba39de431adf4fa8 (127.11.207.129:37735)
I20260812 21:08:48.901142 20614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b622b884f3ff461b93812bbcd8510ce5" candidate_uuid: "6f6fd262a4504405b0b9fce9117ad93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45306934f4024817ba39de431adf4fa8"
I20260812 21:08:48.901197 21114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b622b884f3ff461b93812bbcd8510ce5" candidate_uuid: "6f6fd262a4504405b0b9fce9117ad93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7b5aa69581470993e26ac53e4e2780"
I20260812 21:08:48.901227 20614 raft_consensus.cc:3037] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.901284 21114 raft_consensus.cc:3037] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:48.902228 21114 raft_consensus.cc:2454] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f6fd262a4504405b0b9fce9117ad93a in term 1.
I20260812 21:08:48.902416 20768 leader_election.cc:302] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7b5aa69581470993e26ac53e4e2780, 6f6fd262a4504405b0b9fce9117ad93a; no voters:
I20260812 21:08:48.902415 20614 raft_consensus.cc:2454] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f6fd262a4504405b0b9fce9117ad93a in term 1.
I20260812 21:08:48.902511 21235 raft_consensus.cc:2781] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:48.902683 21235 raft_consensus.cc:686] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 1 LEADER]: Becoming Leader. State: Replica: 6f6fd262a4504405b0b9fce9117ad93a, State: Running, Role: LEADER
I20260812 21:08:48.902781 21235 consensus_queue.cc:237] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } }
I20260812 21:08:48.903628 20394 catalog_manager.cc:5495] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f6fd262a4504405b0b9fce9117ad93a (127.11.207.130). New cstate: current_term: 1 leader_uuid: "6f6fd262a4504405b0b9fce9117ad93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f6fd262a4504405b0b9fce9117ad93a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 39267 } health_report { overall_health: HEALTHY } } }
I20260812 21:08:48.927871 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.927060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.11.207.130:39267)
0812 21:08:48.927109 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:48.927111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:48.927121 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:48.927166 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:48.927167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:48.927182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:48.927239 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:48.927703 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:48.927704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:48.927704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:48.927708 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:48.927723 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:48.927727 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:48.927769 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:48.927803 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 21:08:48.927814 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 21:08:48.929220 20972 tablet.cc:2369] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:48.929792 21114 raft_consensus.cc:1261] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 1 FOLLOWER]: Refusing update from remote peer 6f6fd262a4504405b0b9fce9117ad93a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:48.929791 20614 raft_consensus.cc:1261] T b622b884f3ff461b93812bbcd8510ce5 P 45306934f4024817ba39de431adf4fa8 [term 1 FOLLOWER]: Refusing update from remote peer 6f6fd262a4504405b0b9fce9117ad93a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:48.930143 21235 consensus_queue.cc:1035] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f7b5aa69581470993e26ac53e4e2780" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:48.930446 21238 consensus_queue.cc:1035] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "45306934f4024817ba39de431adf4fa8" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:48.932861 21245 mvcc.cc:205] Tried to move back new op lower bound from 7317786332893601792 to 7317786332786421760. Current Snapshot: MvccSnapshot[applied={T|T < 7317786332893601792}]
I20260812 21:08:48.933267 21248 mvcc.cc:205] Tried to move back new op lower bound from 7317786332893601792 to 7317786332786421760. Current Snapshot: MvccSnapshot[applied={T|T < 7317786332893601792}]
I20260812 21:08:48.935425 20392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:48.935511 20392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:48.936288 20392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f52e2fa4b2be44669c77ac5b030fbb84: Sending DeleteTablet for 3 replicas of tablet b622b884f3ff461b93812bbcd8510ce5
I20260812 21:08:48.936645 20864 tablet_service.cc:1514] Processing DeleteTablet for tablet b622b884f3ff461b93812bbcd8510ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:48 UTC) from {username='slave'} at 127.0.0.1:57962
I20260812 21:08:48.936666 21084 tablet_service.cc:1514] Processing DeleteTablet for tablet b622b884f3ff461b93812bbcd8510ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:48 UTC) from {username='slave'} at 127.0.0.1:52420
I20260812 21:08:48.937182 21266 tablet_replica.cc:330] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: stopping tablet replica
I20260812 21:08:48.937290 21266 raft_consensus.cc:2229] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:48.937276 20613 tablet_service.cc:1514] Processing DeleteTablet for tablet b622b884f3ff461b93812bbcd8510ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:48 UTC) from {username='slave'} at 127.0.0.1:49624
I20260812 21:08:48.937378 21266 raft_consensus.cc:2258] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:48.937461 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20506
I20260812 21:08:48.952764 21266 ts_tablet_manager.cc:1905] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:48.956692 21265 tablet_replica.cc:330] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: stopping tablet replica
I20260812 21:08:48.956857 21265 raft_consensus.cc:2229] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:48.956996 21265 raft_consensus.cc:2258] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:48.957459 21265 ts_tablet_manager.cc:1905] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:48.958854 21265 ts_tablet_manager.cc:1918] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:48.958931 21265 log.cc:1196] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/b622b884f3ff461b93812bbcd8510ce5
I20260812 21:08:48.959168 21265 ts_tablet_manager.cc:1939] T b622b884f3ff461b93812bbcd8510ce5 P 1f7b5aa69581470993e26ac53e4e2780: Deleting consensus metadata
I20260812 21:08:48.959638 20365 catalog_manager.cc:4841] TS 1f7b5aa69581470993e26ac53e4e2780 (127.11.207.131:44939): tablet b622b884f3ff461b93812bbcd8510ce5 (table test-table [id=2ac768992be6415c9a91fc4b2724a8f8]) successfully deleted
I20260812 21:08:48.960614 21266 ts_tablet_manager.cc:1918] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:48.960706 21266 log.cc:1196] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/b622b884f3ff461b93812bbcd8510ce5
I20260812 21:08:48.960970 21266 ts_tablet_manager.cc:1939] T b622b884f3ff461b93812bbcd8510ce5 P 6f6fd262a4504405b0b9fce9117ad93a: Deleting consensus metadata
I20260812 21:08:48.961387 20369 catalog_manager.cc:4841] TS 6f6fd262a4504405b0b9fce9117ad93a (127.11.207.130:39267): tablet b622b884f3ff461b93812bbcd8510ce5 (table test-table [id=2ac768992be6415c9a91fc4b2724a8f8]) successfully deleted
W20260812 21:08:48.963021 20370 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:37735 (error 108)
I20260812 21:08:48.963194 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20749
I20260812 21:08:48.968223 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:48.968074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37735 (local address 127.0.0.1:49628)
0812 21:08:48.968133 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:48.968187 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 21:08:48.968429 20370 catalog_manager.cc:4584] TS 45306934f4024817ba39de431adf4fa8 (127.11.207.129:37735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b622b884f3ff461b93812bbcd8510ce5: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37735: connect: Connection refused (error 111)
I20260812 21:08:48.975771 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20976
I20260812 21:08:49.005749 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20341
2026-08-12T21:08:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (722 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 21:08:49.031139 12094 mini_oidc.cc:152] Starting JWKS server
I20260812 21:08:49.032560 12094 webserver.cc:460] Webserver started at https://127.0.0.1:42293/ using document root <none> and password file <none>
I20260812 21:08:49.032650 12094 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 21:08:49.032831 12094 webserver.cc:460] Webserver started at http://127.0.0.1:37375/ using document root <none> and password file <none>
2026-08-12T21:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:49Z Disabled control of system clock
I20260812 21:08:49.042317 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38423
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:38423
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42293/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/testchainca.cert with env {}
W20260812 21:08:49.126286 21284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:49.126530 21284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:49.126567 21284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:49.127770 21284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:49.127815 21284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:49.127828 21284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:49.127841 21284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:49.127854 21284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:08:49.127867 21284 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:08:49.127879 21284 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42293/jwks.json
I20260812 21:08:49.129489 21284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:38423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42293/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:49.129712 21284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:49.129894 21284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:49.134882 21303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:49.134893 21298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:49.134935 21297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:49.135188 21284 server_base.cc:1034] running on GCE node
I20260812 21:08:49.135465 21284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:49.135758 21284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:49.136926 21284 hybrid_clock.cc:648] HybridClock initialized: now 1786568929136910 us; error 40 us; skew 500 ppm
I20260812 21:08:49.138273 21284 webserver.cc:460] Webserver started at http://127.11.207.190:44477/ using document root <none> and password file <none>
I20260812 21:08:49.138522 21284 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:49.138577 21284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:49.138713 21284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:49.139882 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "ef38c51339854879b8079d235c5eb3ac"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "db095a9efb850af352339a8d6f95cd5c"
server_key_iv: "97e11a3e31971bfd05cb658dc3e289d1"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.140244 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "ef38c51339854879b8079d235c5eb3ac"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "db095a9efb850af352339a8d6f95cd5c"
server_key_iv: "97e11a3e31971bfd05cb658dc3e289d1"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.141575 21284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:49.142364 21316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:49.142513 21284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:49.142612 21284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "ef38c51339854879b8079d235c5eb3ac"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "db095a9efb850af352339a8d6f95cd5c"
server_key_iv: "97e11a3e31971bfd05cb658dc3e289d1"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.142679 21284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:49.155450 21284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:49.155773 21284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:49.155905 21284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:49.159960 21284 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:38423
I20260812 21:08:49.160013 21401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:38423 every 8 connection(s)
I20260812 21:08:49.160377 21284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:49.161433 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:49.163792 21403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac: Bootstrap starting.
I20260812 21:08:49.164363 21403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:49.164624 21403 log.cc:824] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:49.165277 21403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac: No bootstrap required, opened a new log
I20260812 21:08:49.166857 21403 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef38c51339854879b8079d235c5eb3ac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38423 } }
I20260812 21:08:49.166996 21403 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:49.167016 21403 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef38c51339854879b8079d235c5eb3ac, State: Initialized, Role: FOLLOWER
I20260812 21:08:49.167135 21403 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef38c51339854879b8079d235c5eb3ac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38423 } }
I20260812 21:08:49.167215 21403 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:49.167240 21403 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:49.167287 21403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:49.167321 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21284
I20260812 21:08:49.167430 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:49.167583 12094 external_mini_cluster.cc:1428] Setting key f12370b4d1af20d97819b0a745bfe776
I20260812 21:08:49.167915 21403 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef38c51339854879b8079d235c5eb3ac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38423 } }
I20260812 21:08:49.168035 21403 leader_election.cc:302] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef38c51339854879b8079d235c5eb3ac; no voters:
I20260812 21:08:49.168216 21403 leader_election.cc:288] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:49.168318 21411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:49.168511 21403 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:49.168542 21411 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [term 1 LEADER]: Becoming Leader. State: Replica: ef38c51339854879b8079d235c5eb3ac, State: Running, Role: LEADER
I20260812 21:08:49.168771 21411 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef38c51339854879b8079d235c5eb3ac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38423 } }
I20260812 21:08:49.169728 21412 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef38c51339854879b8079d235c5eb3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef38c51339854879b8079d235c5eb3ac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38423 } } }
I20260812 21:08:49.169737 21413 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef38c51339854879b8079d235c5eb3ac. Latest consensus state: current_term: 1 leader_uuid: "ef38c51339854879b8079d235c5eb3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef38c51339854879b8079d235c5eb3ac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38423 } } }
I20260812 21:08:49.169838 21412 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.169842 21413 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.170042 21430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:49.170491 21430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:49.172040 21430 catalog_manager.cc:1347] Generated new cluster ID: e4dffec4c5b64297ab43861310a815a7
I20260812 21:08:49.172094 21430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:49.173961 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.168064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.11.207.190:38423)
0812 21:08:49.168301 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.168312 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.168503 (+ 191us) server_negotiation.cc:407] Connection header received
0812 21:08:49.168630 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.168638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.168712 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.168862 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:49.169147 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.170616 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.171753 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.172375 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.172485 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.173174 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:49.173178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:49.173178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:49.173394 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:49.173435 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:49.173448 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:49.173504 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:49.173597 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 21:08:49.173671 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260812 21:08:49.174969 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:49.191569 21430 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:49.192121 21430 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:49.196849 21430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef38c51339854879b8079d235c5eb3ac: Generated new TSK 0
I20260812 21:08:49.197050 21430 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 21:08:49.221242 21452 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 21:08:49.221413 21452 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 21:08:49.221571 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.205091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.11.207.190:38423)
0812 21:08:49.205159 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.205162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.205374 (+ 212us) server_negotiation.cc:407] Connection header received
0812 21:08:49.205420 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.205421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.205434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.205506 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:08:49.205680 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.206180 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.207314 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.207523 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.207578 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.207799 (+ 221us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:08:49.221292 (+ 13493us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 21:08:49.221501 (+ 209us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43428: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 21:08:49.221629 21410 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43428: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 21:08:49.222270 21450 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:49.205003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38423 (local address 127.0.0.1:43428)
0812 21:08:49.205248 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:49.205260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:49.205336 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:49.205522 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:49.205523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:49.205525 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:08:49.205592 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.205595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.206255 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.206256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.207216 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.207229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.207604 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.207604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.207671 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.207696 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:08:49.221615 (+ 13919us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 21:08:49.221843 (+ 228us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.207.190:38423: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":100,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.207.190:38423: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42293/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 21:08:49.222823 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21284
2026-08-12T21:08:49Z chronyd exiting
I20260812 21:08:49.262781 12094 test_util.cc:175] -----------------------------------------------
I20260812 21:08:49.262863 12094 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568904794702-12094-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 21:08:49.263648 12094 mini_oidc.cc:152] Starting JWKS server
I20260812 21:08:49.264297 12094 webserver.cc:460] Webserver started at https://127.0.0.1:44165/ using document root <none> and password file <none>
I20260812 21:08:49.264369 12094 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 21:08:49.264514 12094 webserver.cc:460] Webserver started at http://127.0.0.1:39893/ using document root <none> and password file <none>
2026-08-12T21:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:49Z Disabled control of system clock
I20260812 21:08:49.272927 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:45059
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:38131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:45059
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44165/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/testchainca.cert with env {}
W20260812 21:08:49.353725 21465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:49.353921 21465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:49.353943 21465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:49.355046 21465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:49.355082 21465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:49.355093 21465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:49.355104 21465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:49.355115 21465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:08:49.355126 21465 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:08:49.355142 21465 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44165/jwks.json
I20260812 21:08:49.356613 21465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:38131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:45059
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:45059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44165/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:49.356835 21465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:49.357010 21465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:49.361634 21482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:49.361616 21479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:49.361788 21465 server_base.cc:1034] running on GCE node
W20260812 21:08:49.361619 21488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:49.362107 21465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:49.362366 21465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:49.363534 21465 hybrid_clock.cc:648] HybridClock initialized: now 1786568929363513 us; error 37 us; skew 500 ppm
I20260812 21:08:49.364881 21465 webserver.cc:460] Webserver started at http://127.11.207.190:43155/ using document root <none> and password file <none>
I20260812 21:08:49.365113 21465 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:49.365155 21465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:49.365281 21465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:49.366304 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "02c06fedcbcc443fb01777348da75eac"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "3638c621b8d35dc58b182b78fed012ed"
server_key_iv: "ea0e2b35e1a1b2136d3df9572aad0da4"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.366631 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "02c06fedcbcc443fb01777348da75eac"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "3638c621b8d35dc58b182b78fed012ed"
server_key_iv: "ea0e2b35e1a1b2136d3df9572aad0da4"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.367934 21465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:49.368625 21498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:49.368844 21465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:49.368932 21465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "02c06fedcbcc443fb01777348da75eac"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "3638c621b8d35dc58b182b78fed012ed"
server_key_iv: "ea0e2b35e1a1b2136d3df9572aad0da4"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.368991 21465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:49.388319 21465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:49.388715 21465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:49.388850 21465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:49.392571 21465 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:45059
I20260812 21:08:49.392616 21582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:45059 every 8 connection(s)
I20260812 21:08:49.392952 21465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:49.393643 21583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:49.396502 21583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac: Bootstrap starting.
I20260812 21:08:49.397074 21583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:49.397357 21583 log.cc:824] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:49.397384 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21465
I20260812 21:08:49.397483 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:49.397675 12094 external_mini_cluster.cc:1428] Setting key 1c12ec0b92f977efa1320152d4fa38c7
I20260812 21:08:49.398248 21583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac: No bootstrap required, opened a new log
I20260812 21:08:49.400220 21583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c06fedcbcc443fb01777348da75eac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45059 } }
I20260812 21:08:49.400395 21583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:49.400444 21583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c06fedcbcc443fb01777348da75eac, State: Initialized, Role: FOLLOWER
I20260812 21:08:49.400571 21583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c06fedcbcc443fb01777348da75eac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45059 } }
I20260812 21:08:49.400636 21583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:49.400666 21583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:49.400713 21583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:49.401314 21583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c06fedcbcc443fb01777348da75eac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45059 } }
I20260812 21:08:49.401412 21583 leader_election.cc:302] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02c06fedcbcc443fb01777348da75eac; no voters:
I20260812 21:08:49.401572 21583 leader_election.cc:288] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:49.401623 21594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:49.401809 21594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [term 1 LEADER]: Becoming Leader. State: Replica: 02c06fedcbcc443fb01777348da75eac, State: Running, Role: LEADER
I20260812 21:08:49.401834 21583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:49.401913 21594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c06fedcbcc443fb01777348da75eac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45059 } }
I20260812 21:08:49.402279 21594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02c06fedcbcc443fb01777348da75eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c06fedcbcc443fb01777348da75eac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45059 } } }
I20260812 21:08:49.402360 21594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.402510 21594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02c06fedcbcc443fb01777348da75eac. Latest consensus state: current_term: 1 leader_uuid: "02c06fedcbcc443fb01777348da75eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02c06fedcbcc443fb01777348da75eac" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45059 } } }
I20260812 21:08:49.402576 21594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.402700 21614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:49.403285 21614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:49.404852 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.398214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.11.207.190:45059)
0812 21:08:49.398474 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.398523 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.398551 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:08:49.399481 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.399490 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.399559 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.399750 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:49.399975 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.401357 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.402494 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.403126 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.403259 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.404107 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:49.404112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:49.404113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:49.404290 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:49.404326 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:49.404337 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:49.404427 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:49.404545 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 21:08:49.404626 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":93,"threads_started":1}
I20260812 21:08:49.405112 21614 catalog_manager.cc:1347] Generated new cluster ID: 7e0c12c17f7e40ab8253a105eae6a05f
I20260812 21:08:49.405170 21614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:49.405807 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:49.412468 21614 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:49.413116 21614 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:49.416765 21614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02c06fedcbcc443fb01777348da75eac: Generated new TSK 0
I20260812 21:08:49.416954 21614 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 21:08:49.438403 21646 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 21:08:49.438500 21646 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 21:08:49.438856 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.423438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.11.207.190:45059)
0812 21:08:49.423495 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.423497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.423537 (+ 40us) server_negotiation.cc:407] Connection header received
0812 21:08:49.423633 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.423634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.423646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.423718 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:08:49.423837 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.424634 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.425467 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.425786 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.425855 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.426056 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:08:49.438603 (+ 12547us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44165/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 21:08:49.438773 (+ 170us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44165/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 21:08:49.438922 21589 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44165/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 21:08:49.438994 21645 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:49.423326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:45059 (local address 127.0.0.1:60210)
0812 21:08:49.423479 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:49.423490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:49.423553 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:49.423720 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:49.423721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:49.423723 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:08:49.423783 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.423785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.424746 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.424748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.425391 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.425394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.425881 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.425882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.425939 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.425946 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:08:49.438737 (+ 12791us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44165/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 21:08:49.438884 (+ 147us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.207.190:45059: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44165/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260812 21:08:49.439451 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21465
2026-08-12T21:08:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 21:08:49.485376 12094 mini_oidc.cc:152] Starting JWKS server
I20260812 21:08:49.486044 12094 webserver.cc:460] Webserver started at https://127.0.0.1:45567/ using document root <none> and password file <none>
I20260812 21:08:49.486088 12094 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 21:08:49.486196 12094 webserver.cc:460] Webserver started at http://127.0.0.1:39831/ using document root <none> and password file <none>
2026-08-12T21:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:49Z Disabled control of system clock
I20260812 21:08:49.494999 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43577
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43577
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45567/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 21:08:49.576814 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:49.577030 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:49.577051 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:49.578192 21658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:49.578228 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:49.578238 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:49.578249 21658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:49.578260 21658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:08:49.578270 21658 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:08:49.578280 21658 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45567/jwks.json
I20260812 21:08:49.579610 21658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45567/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:49.579823 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:49.580072 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:08:49.584825 21658 server_base.cc:1034] running on GCE node
W20260812 21:08:49.584800 21666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:49.584782 21667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:49.584803 21669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:49.585193 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:49.585479 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:49.586619 21658 hybrid_clock.cc:648] HybridClock initialized: now 1786568929586605 us; error 31 us; skew 500 ppm
I20260812 21:08:49.587875 21658 webserver.cc:460] Webserver started at http://127.11.207.190:43013/ using document root <none> and password file <none>
I20260812 21:08:49.588099 21658 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:49.588143 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:49.588276 21658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:49.589361 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "7861032ab1044fc99f9c32fead28482f"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "159c0b815ce524712eb71d578c28789e"
server_key_iv: "f9b0bd37fe5ce67539f11d2849d1d28a"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.589692 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "7861032ab1044fc99f9c32fead28482f"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "159c0b815ce524712eb71d578c28789e"
server_key_iv: "f9b0bd37fe5ce67539f11d2849d1d28a"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.590952 21658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:49.591696 21680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:49.591877 21658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:49.591957 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "7861032ab1044fc99f9c32fead28482f"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "159c0b815ce524712eb71d578c28789e"
server_key_iv: "f9b0bd37fe5ce67539f11d2849d1d28a"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.592026 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:49.612965 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:49.613318 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:49.613476 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:49.617631 21658 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43577
I20260812 21:08:49.617673 21784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43577 every 8 connection(s)
I20260812 21:08:49.618036 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:49.618741 21785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:49.619473 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21658
I20260812 21:08:49.619557 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:49.619709 12094 external_mini_cluster.cc:1428] Setting key 3fb621ab76cf0e5b049d377da60252b4
I20260812 21:08:49.622651 21785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f: Bootstrap starting.
I20260812 21:08:49.623278 21785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:49.623535 21785 log.cc:824] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:49.624181 21785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f: No bootstrap required, opened a new log
I20260812 21:08:49.625304 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.620198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.11.207.190:43577)
0812 21:08:49.620424 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.620432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.620467 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:08:49.620927 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.620937 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.621024 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.621199 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:49.621362 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.622658 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.623452 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.623916 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.624031 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.624701 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:49.624706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:49.624706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:49.624869 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:49.624902 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:49.624912 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:49.624966 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:49.625044 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 21:08:49.625125 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260812 21:08:49.626227 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:49.626277 21785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7861032ab1044fc99f9c32fead28482f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43577 } }
I20260812 21:08:49.626442 21785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:49.626487 21785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7861032ab1044fc99f9c32fead28482f, State: Initialized, Role: FOLLOWER
I20260812 21:08:49.626600 21785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7861032ab1044fc99f9c32fead28482f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43577 } }
I20260812 21:08:49.626675 21785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:49.626713 21785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:49.626760 21785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:49.627455 21785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7861032ab1044fc99f9c32fead28482f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43577 } }
I20260812 21:08:49.627554 21785 leader_election.cc:302] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7861032ab1044fc99f9c32fead28482f; no voters:
I20260812 21:08:49.627730 21785 leader_election.cc:288] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:49.627791 21798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:49.627931 21798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [term 1 LEADER]: Becoming Leader. State: Replica: 7861032ab1044fc99f9c32fead28482f, State: Running, Role: LEADER
I20260812 21:08:49.627969 21785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:49.628022 21798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7861032ab1044fc99f9c32fead28482f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43577 } }
I20260812 21:08:49.628409 21799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7861032ab1044fc99f9c32fead28482f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7861032ab1044fc99f9c32fead28482f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43577 } } }
I20260812 21:08:49.628420 21800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7861032ab1044fc99f9c32fead28482f. Latest consensus state: current_term: 1 leader_uuid: "7861032ab1044fc99f9c32fead28482f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7861032ab1044fc99f9c32fead28482f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43577 } } }
I20260812 21:08:49.628918 21799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.628930 21800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.629179 21821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:49.629678 21821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:49.631011 21821 catalog_manager.cc:1347] Generated new cluster ID: d0ed8b4767624a2c989e55eaae5b152b
I20260812 21:08:49.631059 21821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:49.647630 21821 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:49.648463 21821 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:49.654078 21821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7861032ab1044fc99f9c32fead28482f: Generated new TSK 0
I20260812 21:08:49.654281 21821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:49.663349 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.660252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.11.207.190:43577)
0812 21:08:49.660299 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.660301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.660447 (+ 146us) server_negotiation.cc:407] Connection header received
0812 21:08:49.660516 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.660517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.660529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.660589 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:08:49.660710 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.661212 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.661976 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.662170 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.662223 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.662399 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:08:49.663151 (+ 752us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45567/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 21:08:49.663273 (+ 122us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57690: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45567/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 21:08:49.663410 21795 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57690: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45567/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 21:08:49.663429 21840 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:49.660200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43577 (local address 127.0.0.1:57690)
0812 21:08:49.660382 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:49.660392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:49.660456 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:49.660592 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:49.660593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:49.660594 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:08:49.660654 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.660657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.661290 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.661292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.661903 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.661905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.662236 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.662237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.662309 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.662315 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:08:49.663249 (+ 934us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45567/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 21:08:49.663344 (+ 95us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.207.190:43577: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45567/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 21:08:49.663654 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21658
E20260812 21:08:49.706574 21841 webserver.cc:533] Webserver: sslize failed:
E20260812 21:08:49.706676 21841 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T21:08:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T21:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:49Z Disabled control of system clock
I20260812 21:08:49.743013 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:45549
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:33771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:45549
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/testchainca.cert with env {}
W20260812 21:08:49.828421 21851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:49.828644 21851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:49.828667 21851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:49.829813 21851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:49.829856 21851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:49.829869 21851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:49.829882 21851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:49.829895 21851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:08:49.829907 21851 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:08:49.829918 21851 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 21:08:49.831401 21851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:45549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:45549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:49.831610 21851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:49.831799 21851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:49.836767 21863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:49.836833 21865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:49.836931 21851 server_base.cc:1034] running on GCE node
W20260812 21:08:49.836957 21874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:49.837306 21851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:49.837615 21851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:49.838796 21851 hybrid_clock.cc:648] HybridClock initialized: now 1786568929838777 us; error 34 us; skew 500 ppm
I20260812 21:08:49.840008 21851 webserver.cc:460] Webserver started at http://127.11.207.190:38503/ using document root <none> and password file <none>
I20260812 21:08:49.840256 21851 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:49.840303 21851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:49.840452 21851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:49.841547 21851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "1edee7631e5f4a81a638e99c66a016ba"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "e68ec366815f30c303c7116cc39077fb"
server_key_iv: "1b07a638484e1bc1fa26b5e843c31594"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.841879 21851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "1edee7631e5f4a81a638e99c66a016ba"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "e68ec366815f30c303c7116cc39077fb"
server_key_iv: "1b07a638484e1bc1fa26b5e843c31594"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.843099 21851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:49.843765 21886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:49.843911 21851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:49.843999 21851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "1edee7631e5f4a81a638e99c66a016ba"
format_stamp: "Formatted at 2026-08-12 21:08:49 on dist-test-slave-lkx2"
server_key: "e68ec366815f30c303c7116cc39077fb"
server_key_iv: "1b07a638484e1bc1fa26b5e843c31594"
server_key_version: "encryptionkey@0"
I20260812 21:08:49.844063 21851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:49.857772 21851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:49.858093 21851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:49.858242 21851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:49.862048 21851 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:45549
I20260812 21:08:49.862087 21983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:45549 every 8 connection(s)
I20260812 21:08:49.862423 21851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:49.863395 21984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:49.865753 21984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba: Bootstrap starting.
I20260812 21:08:49.866310 21984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:49.866571 21984 log.cc:824] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:49.867292 21984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba: No bootstrap required, opened a new log
I20260812 21:08:49.867579 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21851
I20260812 21:08:49.867679 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:49.867849 12094 external_mini_cluster.cc:1428] Setting key cca4e94cab751ae929ed3b46e9ba5dd1
I20260812 21:08:49.869573 21984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edee7631e5f4a81a638e99c66a016ba" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45549 } }
I20260812 21:08:49.869777 21984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:49.869823 21984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edee7631e5f4a81a638e99c66a016ba, State: Initialized, Role: FOLLOWER
I20260812 21:08:49.869961 21984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edee7631e5f4a81a638e99c66a016ba" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45549 } }
I20260812 21:08:49.870029 21984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:49.870064 21984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:49.870115 21984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:49.870711 21984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edee7631e5f4a81a638e99c66a016ba" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45549 } }
I20260812 21:08:49.870818 21984 leader_election.cc:302] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1edee7631e5f4a81a638e99c66a016ba; no voters:
I20260812 21:08:49.870994 21984 leader_election.cc:288] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:49.871021 21999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:49.871147 21999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [term 1 LEADER]: Becoming Leader. State: Replica: 1edee7631e5f4a81a638e99c66a016ba, State: Running, Role: LEADER
I20260812 21:08:49.871243 21999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edee7631e5f4a81a638e99c66a016ba" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45549 } }
I20260812 21:08:49.871270 21984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:49.871600 21999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1edee7631e5f4a81a638e99c66a016ba. Latest consensus state: current_term: 1 leader_uuid: "1edee7631e5f4a81a638e99c66a016ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edee7631e5f4a81a638e99c66a016ba" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45549 } } }
I20260812 21:08:49.871698 21999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.871613 22002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1edee7631e5f4a81a638e99c66a016ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edee7631e5f4a81a638e99c66a016ba" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45549 } } }
I20260812 21:08:49.871827 22002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:49.872068 22015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:49.872676 22015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:49.874548 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.868402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.11.207.190:45549)
0812 21:08:49.868612 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.868619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.868645 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:08:49.869125 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.869133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.869198 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.869317 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:49.869577 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.870796 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.871919 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.872563 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.872679 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.873659 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:49.873663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:49.873664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:49.873897 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:49.873946 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:49.873962 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:49.874037 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:49.874249 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 21:08:49.874342 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260812 21:08:49.875275 22015 catalog_manager.cc:1347] Generated new cluster ID: 4572e19cbfb948418348c5f223428b5c
I20260812 21:08:49.875331 22015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:49.875550 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:49.904111 22015 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:49.904714 22015 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:49.907018 22015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1edee7631e5f4a81a638e99c66a016ba: Generated new TSK 0
I20260812 21:08:49.907202 22015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:49.977771 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:49.912876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.11.207.190:45549)
0812 21:08:49.912923 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:49.912925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:49.912979 (+ 54us) server_negotiation.cc:407] Connection header received
0812 21:08:49.913041 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:49.913042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:49.913054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:49.913140 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:08:49.913255 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.914088 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.914851 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.915173 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.915241 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.915392 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:08:49.977516 (+ 62124us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 21:08:49.977695 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59826: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 21:08:49.977859 21997 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59826: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 21:08:49.977905 22052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:49.912781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:45549 (local address 127.0.0.1:59826)
0812 21:08:49.912918 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:49.912928 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:49.912983 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:49.913138 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:49.913139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:49.913140 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:08:49.913191 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.913193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.914163 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.914164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.914769 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:49.914771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:49.915247 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:49.915248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:49.915292 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:49.915298 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:08:49.977668 (+ 62370us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 21:08:49.977805 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.207.190:45549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260812 21:08:49.978256 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21851
2026-08-12T21:08:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (271 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:50 dist-test-slave-lkx2 krb5kdc[22070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:50 dist-test-slave-lkx2 krb5kdc[22070](info): set up 2 sockets
Aug 12 21:08:50 dist-test-slave-lkx2 krb5kdc[22070](info): commencing operation
krb5kdc: starting...
W20260812 21:08:51.408658 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 21:08:51.623402 12094 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.215s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 21:08:51.817756 12094 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.194s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 21:08:51.990859 12094 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.173s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/test-user.keytab.
W20260812 21:08:52.494585 12094 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.504s user 0.000s sys 0.002s
Aug 12 21:08:52 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
W20260812 21:08:53.074834 12094 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.207.190: real 0.557s user 0.001s sys 0.002s
I20260812 21:08:53.075284 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:40579
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:40579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:53.159914 22092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:53.160151 22092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:53.160173 22092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:53.161429 22092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:53.161479 22092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:53.161491 22092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:53.161504 22092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:53.161515 22092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:53.162993 22092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:40579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:40579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:53.163219 22092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:53.163405 22092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:53.168298 22103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:53.168382 22099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.168387 22092 server_base.cc:1034] running on GCE node
W20260812 21:08:53.168334 22100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.168769 22092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:53.169075 22092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:53.170218 22092 hybrid_clock.cc:648] HybridClock initialized: now 1786568933170200 us; error 39 us; skew 500 ppm
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.176154 22092 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:08:53.176656 22092 webserver.cc:460] Webserver started at http://127.11.207.190:36519/ using document root <none> and password file <none>
I20260812 21:08:53.176896 22092 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:53.176945 22092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:53.177075 22092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:53.178160 22092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "b4e59754ee9e4305abceea18fcdce306"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "4eff0714c930acdd1a6bc29a03294536"
server_key_iv: "8ea3d1474ff044eee6418c83ad5f6822"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.178508 22092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "b4e59754ee9e4305abceea18fcdce306"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "4eff0714c930acdd1a6bc29a03294536"
server_key_iv: "8ea3d1474ff044eee6418c83ad5f6822"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.179942 22092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:53.180711 22113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.180866 22092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.180959 22092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "b4e59754ee9e4305abceea18fcdce306"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "4eff0714c930acdd1a6bc29a03294536"
server_key_iv: "8ea3d1474ff044eee6418c83ad5f6822"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.181032 22092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:53.212419 22092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:53.214421 22092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:53.214566 22092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:53.218608 22092 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:40579
I20260812 21:08:53.218649 22220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:40579 every 8 connection(s)
I20260812 21:08:53.219007 22092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:53.219954 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:53.221622 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 22092
I20260812 21:08:53.221732 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:53.221923 12094 external_mini_cluster.cc:1428] Setting key 64d52d3ee31a86f73041e8b029036f1c
I20260812 21:08:53.222687 22221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306: Bootstrap starting.
I20260812 21:08:53.223392 22221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:53.223726 22221 log.cc:824] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:53.224517 22221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306: No bootstrap required, opened a new log
I20260812 21:08:53.226611 22221 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e59754ee9e4305abceea18fcdce306" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40579 } }
I20260812 21:08:53.226801 22221 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:53.226858 22221 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4e59754ee9e4305abceea18fcdce306, State: Initialized, Role: FOLLOWER
I20260812 21:08:53.226992 22221 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e59754ee9e4305abceea18fcdce306" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40579 } }
I20260812 21:08:53.227080 22221 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:53.227110 22221 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:53.227161 22221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:53.227752 22221 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e59754ee9e4305abceea18fcdce306" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40579 } }
I20260812 21:08:53.227854 22221 leader_election.cc:302] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4e59754ee9e4305abceea18fcdce306; no voters:
I20260812 21:08:53.228012 22221 leader_election.cc:288] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:53.228052 22236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:53.228169 22236 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [term 1 LEADER]: Becoming Leader. State: Replica: b4e59754ee9e4305abceea18fcdce306, State: Running, Role: LEADER
I20260812 21:08:53.228283 22221 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:53.228320 22236 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e59754ee9e4305abceea18fcdce306" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40579 } }
I20260812 21:08:53.228685 22236 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4e59754ee9e4305abceea18fcdce306. Latest consensus state: current_term: 1 leader_uuid: "b4e59754ee9e4305abceea18fcdce306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e59754ee9e4305abceea18fcdce306" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40579 } } }
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:53.228921 22236 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:53.228847 22237 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4e59754ee9e4305abceea18fcdce306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e59754ee9e4305abceea18fcdce306" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 40579 } } }
I20260812 21:08:53.229278 22237 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:53.229472 22250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:53.229949 22250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:53.231590 22250 catalog_manager.cc:1347] Generated new cluster ID: 5e5978d9e033422c9f687080008cc49a
I20260812 21:08:53.231633 22250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:53.235297 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.222490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.11.207.190:40579)
0812 21:08:53.222653 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.222661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.222687 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:08:53.223165 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.223172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.223231 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.223347 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.224166 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.225212 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.226219 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.226613 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.226696 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.229253 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.229257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.229258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.229271 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.233642 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.234039 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.234042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.234043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.234162 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.234347 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.234348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.234348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.234674 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.234860 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.234942 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.235089 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260812 21:08:53.255613 22250 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:53.256181 22250 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:53.260154 22250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4e59754ee9e4305abceea18fcdce306: Generated new TSK 0
I20260812 21:08:53.260331 22250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:08:53.295871 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40579
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:53.386049 22268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:53.386274 22268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:53.386310 22268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:53.387485 22268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:53.387526 22268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:53.387560 22268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:53.389195 22268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:53.389446 22268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:53.389662 22268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:53.394884 22284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:53.394901 22281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.394987 22268 server_base.cc:1034] running on GCE node
W20260812 21:08:53.394901 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.395356 22268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:53.395597 22268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:53.396770 22268 hybrid_clock.cc:648] HybridClock initialized: now 1786568933396754 us; error 30 us; skew 500 ppm
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.399292 22268 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:08:53.399689 22268 webserver.cc:460] Webserver started at http://127.11.207.129:33995/ using document root <none> and password file <none>
I20260812 21:08:53.399887 22268 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:53.399926 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:53.400017 22268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:53.401057 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "18c0a5709d30408e8ea6ed4537a5c151"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "4f6cf8275a9d5844831e6fbd8bbffb57"
server_key_iv: "b32c63d12f1dcb1a4f0380e4dca67af8"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.401362 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "18c0a5709d30408e8ea6ed4537a5c151"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "4f6cf8275a9d5844831e6fbd8bbffb57"
server_key_iv: "b32c63d12f1dcb1a4f0380e4dca67af8"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.402587 22268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:53.403403 22294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.403553 22268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:53.403637 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "18c0a5709d30408e8ea6ed4537a5c151"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "4f6cf8275a9d5844831e6fbd8bbffb57"
server_key_iv: "b32c63d12f1dcb1a4f0380e4dca67af8"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.403733 22268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:53.420315 22268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:53.422420 22268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:53.422554 22268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:53.422806 22268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:53.423131 22268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:53.423168 22268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.423198 22268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:53.423215 22268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.429365 22268 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:39993
I20260812 21:08:53.429420 22511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:39993 every 8 connection(s)
I20260812 21:08:53.429780 22268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:53.433871 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 22268
I20260812 21:08:53.433984 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:53.434176 12094 external_mini_cluster.cc:1428] Setting key 6546d20d70b7726ea9344597a195d17d
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:53.439455 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.430568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:47857 (local address 127.11.207.190:40579)
0812 21:08:53.430652 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.430653 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.430824 (+ 171us) server_negotiation.cc:407] Connection header received
0812 21:08:53.431766 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.431768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.431779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.431838 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.432479 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.432991 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.434108 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.434378 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.434454 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.436786 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.436788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.436788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.436797 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.438092 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.438505 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.438507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.438507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.438564 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.438801 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.438802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.438802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.438924 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.438981 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.439265 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.439362 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 21:08:53.439625 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.430519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:40579 (local address 127.11.207.129:47857)
0812 21:08:53.430688 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:53.430732 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:53.431665 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:53.431907 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:53.431910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:53.432176 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:53.432403 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:53.432408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.433054 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.433054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:53.434025 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:53.434029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.434473 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.434474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:53.434615 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.435127 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:53.435137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:53.436686 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:53.438173 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:53.438181 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:53.438183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:53.438427 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:53.438628 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:53.438628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:53.438629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:53.438743 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:53.438984 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:53.438991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:53.439135 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 21:08:53.439235 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 21:08:53.439399 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260812 21:08:53.440094 22512 heartbeater.cc:347] Connected to a master server at 127.11.207.190:40579
I20260812 21:08:53.440191 22512 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:53.440414 22512 heartbeater.cc:511] Master 127.11.207.190:40579 requested a full tablet report, sending...
I20260812 21:08:53.440961 22154 ts_manager.cc:194] Registered new tserver with Master: 18c0a5709d30408e8ea6ed4537a5c151 (127.11.207.129:39993)
I20260812 21:08:53.441650 22154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:47857
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:08:53.460843 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40579
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:53.544183 22519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:53.544451 22519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:53.544492 22519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:53.545663 22519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:53.545708 22519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:53.545744 22519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:53.547277 22519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:53.547528 22519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:53.547761 22519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:53.552757 22525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:53.552752 22532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:53.552870 22524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.552970 22519 server_base.cc:1034] running on GCE node
I20260812 21:08:53.553192 22519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:53.553431 22519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:53.554576 22519 hybrid_clock.cc:648] HybridClock initialized: now 1786568933554554 us; error 39 us; skew 500 ppm
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.557236 22519 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:08:53.557696 22519 webserver.cc:460] Webserver started at http://127.11.207.130:42757/ using document root <none> and password file <none>
I20260812 21:08:53.557932 22519 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:53.557982 22519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:53.558111 22519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:53.559165 22519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "87610ecd3d12486295794358d695a76a"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "1552bfcd22ff32386d76f8ab02b4ce76"
server_key_iv: "aab91228d1f4068573834cc9e15db629"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.559496 22519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "87610ecd3d12486295794358d695a76a"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "1552bfcd22ff32386d76f8ab02b4ce76"
server_key_iv: "aab91228d1f4068573834cc9e15db629"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.560835 22519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:53.561549 22544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.561740 22519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:53.561846 22519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "87610ecd3d12486295794358d695a76a"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "1552bfcd22ff32386d76f8ab02b4ce76"
server_key_iv: "aab91228d1f4068573834cc9e15db629"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.561909 22519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:53.586463 22519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:53.588754 22519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:53.588908 22519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:53.589140 22519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:53.589453 22519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:53.589489 22519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.589519 22519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:53.589543 22519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.595278 22519 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:39231
I20260812 21:08:53.595360 22750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:39231 every 8 connection(s)
I20260812 21:08:53.595649 22519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:53.596477 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 22519
I20260812 21:08:53.596585 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:53.596772 12094 external_mini_cluster.cc:1428] Setting key 3f7895e708d51812475cd281289ee45c
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:53.605191 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.596390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:55115 (local address 127.11.207.190:40579)
0812 21:08:53.596442 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.596444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.596704 (+ 260us) server_negotiation.cc:407] Connection header received
0812 21:08:53.597776 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.597777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.597790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.597838 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.598471 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.598980 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.600050 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.600326 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.600415 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.602498 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.602500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.602500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.602508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.603859 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.604244 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.604245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.604246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.604308 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.604550 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.604551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.604551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.604671 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.604728 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.605002 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.605089 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
I20260812 21:08:53.605356 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.596327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:40579 (local address 127.11.207.130:55115)
0812 21:08:53.596523 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:53.596573 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:53.597685 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:53.597917 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:53.597921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:53.598188 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:53.598383 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:53.598387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.599103 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.599105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:53.599959 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:53.599962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.600440 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.600442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:53.600572 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.601062 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:53.601072 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:53.602414 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:53.603943 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:53.603951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:53.603953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:53.604178 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:53.604380 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:53.604381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:53.604381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:53.604490 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:53.604728 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:53.604735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:53.604882 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 21:08:53.604972 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:08:53.605138 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260812 21:08:53.605785 22751 heartbeater.cc:347] Connected to a master server at 127.11.207.190:40579
I20260812 21:08:53.605881 22751 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:53.606068 22751 heartbeater.cc:511] Master 127.11.207.190:40579 requested a full tablet report, sending...
I20260812 21:08:53.606654 22154 ts_manager.cc:194] Registered new tserver with Master: 87610ecd3d12486295794358d695a76a (127.11.207.130:39231)
I20260812 21:08:53.607072 22154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:55115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:08:53.623155 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40579
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:53.706259 22759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:53.706456 22759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:53.706478 22759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:53.707662 22759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:53.707708 22759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:53.707741 22759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:53.709307 22759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:40579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:53.709535 22759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:53.709762 22759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:53.714803 22774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:53.714799 22766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.714919 22759 server_base.cc:1034] running on GCE node
W20260812 21:08:53.714798 22769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:53.715240 22759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:53.715504 22759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:53.716709 22759 hybrid_clock.cc:648] HybridClock initialized: now 1786568933716697 us; error 32 us; skew 500 ppm
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.719188 22759 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:08:53.719576 22759 webserver.cc:460] Webserver started at http://127.11.207.131:34191/ using document root <none> and password file <none>
I20260812 21:08:53.719782 22759 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:53.719827 22759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:53.719920 22759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:53.720966 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "ca6b29571a8c4d58a26c069ed473c5d1"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "8ec106ba925523c50a4983cceb3e6400"
server_key_iv: "102d7adbd99313d0bcdec80c1e6b0d7b"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.721261 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "ca6b29571a8c4d58a26c069ed473c5d1"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "8ec106ba925523c50a4983cceb3e6400"
server_key_iv: "102d7adbd99313d0bcdec80c1e6b0d7b"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.722527 22759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:53.723212 22789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.723371 22759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:53.723451 22759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ca6b29571a8c4d58a26c069ed473c5d1"
format_stamp: "Formatted at 2026-08-12 21:08:53 on dist-test-slave-lkx2"
server_key: "8ec106ba925523c50a4983cceb3e6400"
server_key_iv: "102d7adbd99313d0bcdec80c1e6b0d7b"
server_key_version: "encryptionkey@0"
I20260812 21:08:53.723510 22759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:53.738164 22759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:53.740398 22759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:53.740517 22759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:53.740727 22759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:53.741001 22759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:53.741031 22759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.741060 22759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:53.741074 22759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:53.747093 22759 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:33679
I20260812 21:08:53.747159 23007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:33679 every 8 connection(s)
I20260812 21:08:53.747460 22759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:53.748871 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 22759
I20260812 21:08:53.748940 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:53.749084 12094 external_mini_cluster.cc:1428] Setting key a4eb2c90b87f09ef2063a9e6c1144e2a
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:08:53.757050 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.748202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:52657 (local address 127.11.207.190:40579)
0812 21:08:53.748265 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.748267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.748516 (+ 249us) server_negotiation.cc:407] Connection header received
0812 21:08:53.749584 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.749585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.749597 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.749656 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.750372 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.750887 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.751853 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.752045 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.752098 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.754212 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.754213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.754213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.754221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.755608 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.756000 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.756001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.756001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.756060 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.756392 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.756394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.756394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.756532 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.756600 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.756904 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.756978 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:08:53.757221 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.748161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:40579 (local address 127.11.207.131:52657)
0812 21:08:53.748334 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:53.748437 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:53.749489 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:53.749746 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:53.749751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:53.750070 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:53.750284 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:53.750289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.750970 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.750970 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:53.751782 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:53.751784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.752128 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.752129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:53.752253 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.752748 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:53.752757 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:53.754110 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:53.755690 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:53.755697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:53.755699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:53.755930 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:53.756142 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:53.756143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:53.756144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:53.756277 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:53.756618 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:53.756625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:53.756761 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 21:08:53.756864 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 21:08:53.756995 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260812 21:08:53.757697 23011 heartbeater.cc:347] Connected to a master server at 127.11.207.190:40579
I20260812 21:08:53.757797 23011 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:53.757969 23011 heartbeater.cc:511] Master 127.11.207.190:40579 requested a full tablet report, sending...
I20260812 21:08:53.758333 22154 ts_manager.cc:194] Registered new tserver with Master: ca6b29571a8c4d58a26c069ed473c5d1 (127.11.207.131:33679)
I20260812 21:08:53.758730 22154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:52657
I20260812 21:08:53.760901 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/test-admin.keytab.
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.777379 12094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 21:08:53.779120 23024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:53.778251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.11.207.129:39993)
0812 21:08:53.778409 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.778414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.778423 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:53.778447 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.778449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.778462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.778527 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.779034 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55814: BlockingRecv error: recv got EOF from 127.0.0.1:55814 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
W20260812 21:08:53.779162 12094 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc' of type FILE
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.780217 12094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.781324 12094 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:53.790098 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.781888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55828 (local address 127.11.207.129:39993)
0812 21:08:53.781930 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.781932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.782012 (+ 80us) server_negotiation.cc:407] Connection header received
0812 21:08:53.782111 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.782111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.782122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.782203 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.782713 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.783897 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.784799 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.785333 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.785402 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.787505 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.787513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.787514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.787521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.789038 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.789389 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.789392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.789392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.789443 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.789630 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.789630 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.789631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.789787 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.789926 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.789959 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.790021 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 21:08:53.792011 23024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:53.791461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.11.207.129:39993)
0812 21:08:53.791501 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.791502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.791615 (+ 113us) server_negotiation.cc:407] Connection header received
0812 21:08:53.791639 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.791639 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.791651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.791713 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.791958 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55830: BlockingRecv error: recv got EOF from 127.0.0.1:55830 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 21:08:53.792085 12094 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc' of type FILE
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.793056 12094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.794087 12094 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:53.800751 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.794657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55844 (local address 127.11.207.129:39993)
0812 21:08:53.794738 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.794739 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.794748 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:53.794771 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.794772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.794782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.794829 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.795166 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.795643 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.796325 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.796565 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.796616 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.798565 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.798566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.798566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.798573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.799791 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.800075 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.800076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.800076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.800137 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.800296 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.800296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.800297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.800420 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.800498 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.800606 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.800674 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 21:08:53.802168 23024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:53.801632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.11.207.129:39993)
0812 21:08:53.801670 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.801672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.801736 (+ 64us) server_negotiation.cc:407] Connection header received
0812 21:08:53.801805 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.801806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.801815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.801859 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.802122 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55852: BlockingRecv error: recv got EOF from 127.0.0.1:55852 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 21:08:53.802268 12094 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568904794702-12094-0/krb5kdc/krb5cc' of type FILE
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.803236 12094 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:53.804013 12094 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[22070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:08:53.810547 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:53.804539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.11.207.129:39993)
0812 21:08:53.804584 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:53.804586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:53.804642 (+ 56us) server_negotiation.cc:407] Connection header received
0812 21:08:53.804691 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:53.804692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:53.804701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:53.804747 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:53.805076 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.805535 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.806215 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:53.806416 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:53.806468 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:53.808468 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:53.808469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:53.808469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:53.808475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:53.809718 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.810011 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.810012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.810012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.810057 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:53.810187 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:53.810187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:53.810188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:53.810268 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:53.810348 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:53.810405 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:08:53.810460 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:53.811178 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 22268
I20260812 21:08:53.835703 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 22519
I20260812 21:08:53.853215 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 22759
I20260812 21:08:53.868003 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 22092
2026-08-12T21:08:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3704 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[23048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[23048](info): set up 2 sockets
Aug 12 21:08:53 dist-test-slave-lkx2 krb5kdc[23048](info): commencing operation
krb5kdc: starting...
W20260812 21:08:55.088979 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab.
Entry for principal oryx/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab.
I20260812 21:08:55.213533 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34819
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab
--principal=oryx/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:55.295692 23062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:55.295925 23062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:55.295958 23062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:55.297185 23062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:55.297228 23062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:55.297246 23062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:55.297263 23062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:55.297278 23062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:55.298802 23062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab
--principal=oryx/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:55.299049 23062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:55.299258 23062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:55.304203 23080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:55.304211 23078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:55.304265 23077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:55.304303 23062 server_base.cc:1034] running on GCE node
I20260812 21:08:55.304718 23062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:55.305006 23062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:55.306155 23062 hybrid_clock.cc:648] HybridClock initialized: now 1786568935306129 us; error 40 us; skew 500 ppm
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:55.308758 23062 init.cc:377] Logged in from keytab as oryx/127.11.207.190@KRBTEST.COM (short username oryx)
I20260812 21:08:55.309226 23062 webserver.cc:460] Webserver started at http://127.11.207.190:33899/ using document root <none> and password file <none>
I20260812 21:08:55.309443 23062 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:55.309494 23062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:55.309628 23062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:55.310727 23062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "286919065e18466f8495e6f3dae8c02a"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "ea0768d8166a0c4087be8353d39d13a4"
server_key_iv: "2f32ab1f72ba0898de5c33d5040c55c3"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.311085 23062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "286919065e18466f8495e6f3dae8c02a"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "ea0768d8166a0c4087be8353d39d13a4"
server_key_iv: "2f32ab1f72ba0898de5c33d5040c55c3"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.312434 23062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:08:55.313122 23093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.313267 23062 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:55.313355 23062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "286919065e18466f8495e6f3dae8c02a"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "ea0768d8166a0c4087be8353d39d13a4"
server_key_iv: "2f32ab1f72ba0898de5c33d5040c55c3"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.313438 23062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:55.331965 23062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:55.334264 23062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:55.334426 23062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:55.338670 23062 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:34819
I20260812 21:08:55.338732 23191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:34819 every 8 connection(s)
I20260812 21:08:55.339082 23062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:55.340078 23192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:55.342536 23192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Bootstrap starting.
I20260812 21:08:55.343115 23192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:55.343369 23192 log.cc:824] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:55.344132 23192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: No bootstrap required, opened a new log
I20260812 21:08:55.345854 23192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } }
I20260812 21:08:55.346001 23192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:55.346025 23192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 286919065e18466f8495e6f3dae8c02a, State: Initialized, Role: FOLLOWER
I20260812 21:08:55.346143 23192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } }
I20260812 21:08:55.346226 23192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:55.346256 23192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:55.346290 23192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:55.347076 23192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } }
I20260812 21:08:55.347167 23192 leader_election.cc:302] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 286919065e18466f8495e6f3dae8c02a; no voters:
I20260812 21:08:55.347303 23192 leader_election.cc:288] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:55.347373 23199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:55.347539 23199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 LEADER]: Becoming Leader. State: Replica: 286919065e18466f8495e6f3dae8c02a, State: Running, Role: LEADER
I20260812 21:08:55.347623 23192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:55.347636 23199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } }
I20260812 21:08:55.348021 23200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } }
I20260812 21:08:55.348040 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 286919065e18466f8495e6f3dae8c02a. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } }
I20260812 21:08:55.348201 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:55.348196 23200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:55.348621 23218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:55.349123 23218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:55.349265 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 23062
I20260812 21:08:55.349362 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:08:55.349548 12094 external_mini_cluster.cc:1428] Setting key c02d42f23c40266aad94a979f9b7398e
I20260812 21:08:55.350875 23218 catalog_manager.cc:1347] Generated new cluster ID: c502e73b744e4a8da3609363b00bb5eb
I20260812 21:08:55.350930 23218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:08:55.358743 23218 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:55.359404 23218 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:55.361474 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.350006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.11.207.190:34819)
0812 21:08:55.350196 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:55.350205 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:55.350241 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:08:55.350402 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:55.350410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:55.350484 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:55.350630 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:55.351425 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.352842 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.353932 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.354542 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.354639 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.357440 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:55.357445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:55.357446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:55.357461 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:55.359481 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.359883 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.359887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.359887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.359990 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.360161 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.360162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.360162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.360474 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:55.361009 (+ 535us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:55.361079 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 21:08:55.361221 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260812 21:08:55.364773 23218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Generated new TSK 0
I20260812 21:08:55.364940 23218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab.
Entry for principal oryx/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab.
I20260812 21:08:55.396935 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34819
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:55.480237 23236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:55.480463 23236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:55.480487 23236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:55.481642 23236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:55.481683 23236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:55.481709 23236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:55.483181 23236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:55.483415 23236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:55.483636 23236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:55.488695 23259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:55.488799 23236 server_base.cc:1034] running on GCE node
W20260812 21:08:55.488833 23249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:55.488708 23253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:55.489164 23236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:55.489414 23236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:55.490566 23236 hybrid_clock.cc:648] HybridClock initialized: now 1786568935490551 us; error 28 us; skew 500 ppm
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:55.493032 23236 init.cc:377] Logged in from keytab as oryx/127.11.207.129@KRBTEST.COM (short username oryx)
I20260812 21:08:55.493453 23236 webserver.cc:460] Webserver started at http://127.11.207.129:38973/ using document root <none> and password file <none>
I20260812 21:08:55.493652 23236 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:55.493700 23236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:55.493824 23236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:55.494858 23236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "b3078d23e0d34fe5b5011d6c50e1da9c"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "44c5bb0f418f1049627698b4079cf408"
server_key_iv: "f628fc17f9efff527c56545426a4a15a"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.495191 23236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "b3078d23e0d34fe5b5011d6c50e1da9c"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "44c5bb0f418f1049627698b4079cf408"
server_key_iv: "f628fc17f9efff527c56545426a4a15a"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.496451 23236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:55.497157 23275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.497304 23236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:55.497390 23236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "b3078d23e0d34fe5b5011d6c50e1da9c"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "44c5bb0f418f1049627698b4079cf408"
server_key_iv: "f628fc17f9efff527c56545426a4a15a"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.497458 23236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:55.524761 23236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:55.526576 23236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:55.526715 23236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:55.526969 23236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:55.527271 23236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:55.527305 23236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.527335 23236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:55.527354 23236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.533041 23236 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:40121
I20260812 21:08:55.533111 23473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:40121 every 8 connection(s)
I20260812 21:08:55.533425 23236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:08:55.542078 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.534150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:48103 (local address 127.11.207.190:34819)
0812 21:08:55.534229 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:55.534232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:55.534394 (+ 162us) server_negotiation.cc:407] Connection header received
0812 21:08:55.534894 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:55.534894 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:55.534904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:55.534955 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:55.535872 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.536393 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.537338 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.537546 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.537599 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.539818 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:55.539819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:55.539820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:55.539827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:55.540988 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.541342 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.541343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.541343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.541388 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.541570 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.541570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.541570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.541665 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:55.541709 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:55.541941 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 21:08:55.542013 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:08:55.542218 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.534113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:48103)
0812 21:08:55.534271 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:55.534309 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:55.534796 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:55.535011 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:55.535014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:55.535585 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:55.535780 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:55.535785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.536451 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.536451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:55.537256 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:55.537259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.537641 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.537642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:55.537806 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.538331 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:55.538340 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:55.539717 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:55.541056 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:55.541063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:55.541065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:55.541283 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:55.541434 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:55.541435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:55.541436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:55.541529 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:55.541705 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:55.541708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:55.541839 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 21:08:55.541918 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 21:08:55.542036 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260812 21:08:55.542624 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 23236
I20260812 21:08:55.542721 23474 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34819
I20260812 21:08:55.542747 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:08:55.542829 23474 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:55.542948 12094 external_mini_cluster.cc:1428] Setting key 6eef91256ba53a63485cb29e2db6de22
I20260812 21:08:55.543009 23474 heartbeater.cc:511] Master 127.11.207.190:34819 requested a full tablet report, sending...
I20260812 21:08:55.543581 23122 ts_manager.cc:194] Registered new tserver with Master: b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121)
I20260812 21:08:55.544171 23122 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:48103
WARNING: no policy specified for oryx/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab.
Entry for principal oryx/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab.
I20260812 21:08:55.568686 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34819
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:55.654088 23483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:55.654318 23483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:55.654343 23483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:55.655504 23483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:55.655551 23483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:55.655583 23483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:55.657096 23483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:55.657320 23483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:55.657647 23483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:55.662536 23497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:55.662555 23501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:55.662623 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:55.662879 23483 server_base.cc:1034] running on GCE node
I20260812 21:08:55.663071 23483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:55.663290 23483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:55.664403 23483 hybrid_clock.cc:648] HybridClock initialized: now 1786568935664388 us; error 29 us; skew 500 ppm
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:55.666744 23483 init.cc:377] Logged in from keytab as oryx/127.11.207.130@KRBTEST.COM (short username oryx)
I20260812 21:08:55.667183 23483 webserver.cc:460] Webserver started at http://127.11.207.130:33149/ using document root <none> and password file <none>
I20260812 21:08:55.667397 23483 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:55.667443 23483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:55.667572 23483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:55.668664 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "bdbffa9bc5594328be86882ff6e48ffa"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "3944b29f81c2c198f1ebe68daacbb344"
server_key_iv: "e8fadb9bd9f9ca80e9f935465ffaa200"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.669003 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "bdbffa9bc5594328be86882ff6e48ffa"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "3944b29f81c2c198f1ebe68daacbb344"
server_key_iv: "e8fadb9bd9f9ca80e9f935465ffaa200"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.670244 23483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:55.670953 23518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.671103 23483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:55.671186 23483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "bdbffa9bc5594328be86882ff6e48ffa"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "3944b29f81c2c198f1ebe68daacbb344"
server_key_iv: "e8fadb9bd9f9ca80e9f935465ffaa200"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.671259 23483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:55.686939 23483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:55.688771 23483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:55.688905 23483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:55.689121 23483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:55.689445 23483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:55.689479 23483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.689508 23483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:55.689532 23483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.695248 23483 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:35089
I20260812 21:08:55.695307 23755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:35089 every 8 connection(s)
I20260812 21:08:55.695617 23483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:08:55.703987 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.696313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:42179 (local address 127.11.207.190:34819)
0812 21:08:55.696397 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:55.696399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:55.696629 (+ 230us) server_negotiation.cc:407] Connection header received
0812 21:08:55.697111 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:55.697112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:55.697123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:55.697170 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:55.698062 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.698551 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.699497 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.699681 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.699732 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.701877 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:55.701878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:55.701879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:55.701885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:55.702926 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.703280 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.703281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.703281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.703329 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.703509 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.703509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.703510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.703606 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:55.703648 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:55.703862 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 21:08:55.703922 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:08:55.704162 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.696289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:42179)
0812 21:08:55.696505 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:55.696544 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:55.697032 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:55.697234 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:55.697237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:55.697801 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:55.697998 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:55.698004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.698611 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.698612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:55.699427 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:55.699430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.699769 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.699770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:55.699867 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.700311 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:55.700320 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:55.701790 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:55.702994 (+ 1204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:55.703001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:55.703003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:55.703222 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:55.703377 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:55.703377 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:55.703378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:55.703468 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:55.703647 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:55.703650 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:55.703762 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 21:08:55.703845 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 21:08:55.703964 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":121,"threads_started":1}
I20260812 21:08:55.704452 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 23483
I20260812 21:08:55.704552 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:08:55.704689 23756 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34819
I20260812 21:08:55.704751 12094 external_mini_cluster.cc:1428] Setting key 136e98b5abe8ebb2dbc1cca780e1996e
I20260812 21:08:55.704778 23756 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:55.705015 23756 heartbeater.cc:511] Master 127.11.207.190:34819 requested a full tablet report, sending...
I20260812 21:08:55.705410 23126 ts_manager.cc:194] Registered new tserver with Master: bdbffa9bc5594328be86882ff6e48ffa (127.11.207.130:35089)
I20260812 21:08:55.705883 23126 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:42179
WARNING: no policy specified for oryx/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab.
Entry for principal oryx/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab.
I20260812 21:08:55.730615 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34819
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:55.817188 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:55.817387 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:55.817420 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:55.818538 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:55.818581 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:55.818616 23761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:55.820047 23761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:34819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:55.820283 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:55.820511 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:55.825320 23770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:55.825389 23761 server_base.cc:1034] running on GCE node
W20260812 21:08:55.825439 23772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:55.825320 23774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:55.825870 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:55.826087 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:55.827232 23761 hybrid_clock.cc:648] HybridClock initialized: now 1786568935827218 us; error 27 us; skew 500 ppm
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:55.829766 23761 init.cc:377] Logged in from keytab as oryx/127.11.207.131@KRBTEST.COM (short username oryx)
I20260812 21:08:55.830190 23761 webserver.cc:460] Webserver started at http://127.11.207.131:37295/ using document root <none> and password file <none>
I20260812 21:08:55.830467 23761 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:55.830526 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:55.830698 23761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:55.832093 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "6fb3fbaa178a46ae846877d15304b001"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "868d9ff197cb13beb0589ed31f7cafdc"
server_key_iv: "00ac077fdcf7c65c280c5b03c7278f79"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.832574 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "6fb3fbaa178a46ae846877d15304b001"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "868d9ff197cb13beb0589ed31f7cafdc"
server_key_iv: "00ac077fdcf7c65c280c5b03c7278f79"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.834314 23761 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:55.835258 23784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.835469 23761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:55.835551 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "6fb3fbaa178a46ae846877d15304b001"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "868d9ff197cb13beb0589ed31f7cafdc"
server_key_iv: "00ac077fdcf7c65c280c5b03c7278f79"
server_key_version: "encryptionkey@0"
I20260812 21:08:55.835621 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:55.857187 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:55.859333 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:55.859443 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:55.859647 23761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:55.859952 23761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:55.859988 23761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.860029 23761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:55.860052 23761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:55.865824 23761 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:42515
I20260812 21:08:55.865890 24019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:42515 every 8 connection(s)
I20260812 21:08:55.866237 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:08:55 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:08:55.874670 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.866926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:60799 (local address 127.11.207.190:34819)
0812 21:08:55.866976 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:55.866978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:55.867162 (+ 184us) server_negotiation.cc:407] Connection header received
0812 21:08:55.867633 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:55.867634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:55.867645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:55.867692 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:55.868652 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.869160 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.870126 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.870313 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.870363 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.872287 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:55.872288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:55.872289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:55.872297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:55.873559 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.873942 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.873943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.873943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.873998 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:55.874179 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:55.874179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:55.874179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:55.874277 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:55.874323 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:55.874543 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 21:08:55.874603 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:55.874929 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:55.866901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:60799)
0812 21:08:55.867048 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:55.867084 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:55.867558 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:55.867781 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:55.867785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:55.868365 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:55.868579 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:55.868585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.869226 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.869227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:55.870061 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:55.870064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:55.870372 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:55.870372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:55.870469 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:55.870880 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:55.870890 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:55.872192 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:55.873616 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:55.873624 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:55.873626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:55.873880 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:55.874041 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:55.874041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:55.874042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:55.874138 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:55.874324 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:55.874327 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:55.874445 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 21:08:55.874522 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 21:08:55.874674 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260812 21:08:55.875447 24020 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34819
I20260812 21:08:55.875530 24020 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:55.875701 24020 heartbeater.cc:511] Master 127.11.207.190:34819 requested a full tablet report, sending...
I20260812 21:08:55.876094 23126 ts_manager.cc:194] Registered new tserver with Master: 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515)
I20260812 21:08:55.876137 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 23761
I20260812 21:08:55.876219 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:08:55.876438 12094 external_mini_cluster.cc:1428] Setting key aca7b5dbbde139949a72b4f9355685f6
I20260812 21:08:55.876600 23126 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:60799
I20260812 21:08:55.877301 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.207.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.189.keytab.
Entry for principal oryx/127.11.207.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.189.keytab.
I20260812 21:08:55.903144 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:37091
--webserver_interface=127.11.207.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.189.keytab
--principal=oryx/127.11.207.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34819,127.11.207.189:37091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:55.994249 24030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:55.994462 24030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:55.994491 24030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:55.995615 24030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:55.995654 24030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:55.995682 24030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:55.995698 24030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:55.995716 24030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:55.997292 24030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34819,127.11.207.189:37091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.189.keytab
--principal=oryx/127.11.207.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:37091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.207.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
W20260812 21:08:55.997412 24030 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.207.190:34819,127.11.207.189:37091'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 21:08:55.997553 24030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:55.997762 24030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:56.002609 24038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.002722 24030 server_base.cc:1034] running on GCE node
W20260812 21:08:56.002635 24041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.002635 24047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.003041 24030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.003338 24030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.004479 24030 hybrid_clock.cc:648] HybridClock initialized: now 1786568936004456 us; error 34 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.006942 24030 init.cc:377] Logged in from keytab as oryx/127.11.207.189@KRBTEST.COM (short username oryx)
I20260812 21:08:56.007381 24030 webserver.cc:460] Webserver started at http://127.11.207.189:33569/ using document root <none> and password file <none>
I20260812 21:08:56.007573 24030 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.007613 24030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.007711 24030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:56.008818 24030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/instance:
uuid: "f48f780f144f4656959ad5e5fc9f084b"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "47f570351c430b8aaeae6e5361c3857a"
server_key_iv: "28faada2b145de9f1fc4af44ae9da268"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.009214 24030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal/instance:
uuid: "f48f780f144f4656959ad5e5fc9f084b"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "47f570351c430b8aaeae6e5361c3857a"
server_key_iv: "28faada2b145de9f1fc4af44ae9da268"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.010568 24030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:56.011296 24070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.011461 24030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:56.011538 24030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
uuid: "f48f780f144f4656959ad5e5fc9f084b"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "47f570351c430b8aaeae6e5361c3857a"
server_key_iv: "28faada2b145de9f1fc4af44ae9da268"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.011602 24030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.027875 24030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.030174 24030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:56.030316 24030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.035631 24030 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.189:37091
I20260812 21:08:56.035681 24183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.189:37091 every 8 connection(s)
I20260812 21:08:56.036037 24030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
I20260812 21:08:56.037240 24186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:56.038971 24186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:56.039242 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 24030
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.189@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
I20260812 21:08:56.049284 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.039769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.11.207.190:34819)
0812 21:08:56.039825 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.039827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.040144 (+ 317us) server_negotiation.cc:407] Connection header received
0812 21:08:56.040854 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.040856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.040868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.040924 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.041627 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.042415 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.043617 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.043838 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.043902 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.046321 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.046323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.046323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.046333 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.047821 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.048240 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.048241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.048242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.048302 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.048539 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.048540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.048540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.048697 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.048768 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.049067 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.049191 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:08:56.049481 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.040002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.11.207.189:37091)
0812 21:08:56.040168 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.040173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.040210 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:08:56.040758 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.040766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.040789 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.040909 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.041631 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.042795 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.043757 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.044132 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.044181 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.046788 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.046797 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.046797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.046805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.048295 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.048648 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.048651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.048651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.048703 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.048911 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.048912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.048912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.049084 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.049216 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.049305 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.049386 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260812 21:08:56.049548 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.039786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33290)
0812 21:08:56.040004 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.040051 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.040756 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.040967 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.040969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.041290 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.041538 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.041543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.042526 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.042527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.043530 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.043533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.043914 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.043915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.044020 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.044535 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.044546 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.046207 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.047889 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.047897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.047899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.048169 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.048371 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.048372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.048372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.048482 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.048757 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.048763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.048940 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.049024 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.049181 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260812 21:08:56.050218 24186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.189@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
I20260812 21:08:56.055639 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.050454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42362)
0812 21:08:56.050503 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.050513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.050557 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.050647 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.050648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.050760 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.050793 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.050794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.051459 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.051460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.051948 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.051950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.052248 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.052249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.052287 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.052627 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.052631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.053714 (+ 1083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.054889 (+ 1175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.054890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.054891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.055053 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.055199 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.055199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.055199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.055245 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.055410 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.055410 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.055442 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.055485 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.055558 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:08:56.055719 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.050493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.11.207.189:37091)
0812 21:08:56.050523 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.050525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.050547 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:08:56.050598 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.050599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.050607 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.050643 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.050860 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.051319 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.052003 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.052201 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.052243 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.053815 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.053817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.053817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.053823 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.054791 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.055111 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.055112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.055112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.055155 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.055286 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.055286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.055286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.055371 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.055413 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.055501 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.055597 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":14}
I20260812 21:08:56.057327 24186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Bootstrap starting.
I20260812 21:08:56.058214 24186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:56.058576 24186 log.cc:824] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:56.059353 24186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: No bootstrap required, opened a new log
I20260812 21:08:56.061232 24186 raft_consensus.cc:348] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } }
I20260812 21:08:56.061357 24186 raft_consensus.cc:374] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:56.061385 24186 raft_consensus.cc:729] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f48f780f144f4656959ad5e5fc9f084b, State: Initialized, Role: FOLLOWER
I20260812 21:08:56.061511 24186 consensus_queue.cc:260] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } }
I20260812 21:08:56.061832 24186 sys_catalog.cc:564] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:56.061947 24197 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } } }
I20260812 21:08:56.062041 24197 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:08:56.063607 24213 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:56.063665 24213 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:56.065423 24030 master_runner.cc:428] Detected that this master f48f780f144f4656959ad5e5fc9f084b is joining an existing cluster
I20260812 21:08:56.065493 24030 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.207.189:37091
I20260812 21:08:56.073764 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.066561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.11.207.190:34819)
0812 21:08:56.066616 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.066618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.066849 (+ 231us) server_negotiation.cc:407] Connection header received
0812 21:08:56.066897 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.066898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.066910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.066977 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.067534 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.068304 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.069364 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.069564 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.069617 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.070809 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.070811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.070811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.070819 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.072393 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.072802 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.072804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.072804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.072880 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.073097 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.073098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.073099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.073297 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.073348 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.073589 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.073668 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:56.074126 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.066503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33300)
0812 21:08:56.066780 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.066791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.066854 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.066979 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.066979 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.067142 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.067455 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.067457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.068437 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.068438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.069288 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.069291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.069633 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.069633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.069780 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.070264 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.070270 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.070694 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.072489 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.072490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.072490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.072713 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.072941 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.072942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.072943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.073010 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.073412 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.073414 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.073458 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.073538 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.073688 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":213,"threads_started":1}
I20260812 21:08:56.074761 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.066597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42366)
0812 21:08:56.068439 (+ 1842us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.068450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.068514 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.068643 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.068643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.068857 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.068960 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.068962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.069804 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.069805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.070706 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.070708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.071176 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.071177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.071246 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.071566 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.071571 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.071922 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.073503 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.073504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.073504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.073722 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.073895 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.073896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.073896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.073943 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.074154 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.074155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.074206 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.074279 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.074488 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1825,"thread_start_us":163,"threads_started":1}
I20260812 21:08:56.075037 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.066550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.11.207.189:37091)
0812 21:08:56.066646 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.066647 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.068523 (+ 1876us) server_negotiation.cc:407] Connection header received
0812 21:08:56.068569 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.068570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.068579 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.068633 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.069041 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.069701 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.070787 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.071075 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.071150 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.071998 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.071999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.072000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.072006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.073438 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.073788 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.073789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.073789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.073837 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.073999 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.074000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.074001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.074101 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.074154 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.074405 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.074964 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":518,"server-negotiator.queue_time_us":79}
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.190@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
I20260812 21:08:56.083745 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.076541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42376)
0812 21:08:56.076681 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.076735 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.076809 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.076938 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.076941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.077283 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.077347 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.077350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.077921 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.077922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.078855 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.078858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.079144 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.079145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.079237 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.079622 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.079627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.081175 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.082520 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.082530 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.082532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.082735 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.083053 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.083054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.083054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.083131 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.083364 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.083374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.083437 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.083510 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.083640 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260812 21:08:56.083735 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.076603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.11.207.189:37091)
0812 21:08:56.076643 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.076644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.076786 (+ 142us) server_negotiation.cc:407] Connection header received
0812 21:08:56.076882 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.076882 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.076889 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.076932 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.077412 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.077857 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.078923 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.079088 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.079131 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.081257 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.081259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.081259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.081266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.082443 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.082803 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.082804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.082804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.082956 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.083188 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.083189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.083190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.083304 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.083362 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.083543 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.083627 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:56.084442 23122 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.207.189:37091: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b"
member_type: NON_VOTER
last_known_addr {
host: "127.11.207.189"
port: 37091
}
attrs {
promote: true
}
}
dest_uuid: "286919065e18466f8495e6f3dae8c02a"
cas_config_opid_index: -1
I20260812 21:08:56.084856 23122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } }
I20260812 21:08:56.085430 24234 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f48f780f144f4656959ad5e5fc9f084b (127.11.207.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } }
I20260812 21:08:56.085877 24122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:56.086651 24234 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.207.189:37091
I20260812 21:08:56.086666 24122 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Refusing update from remote peer 286919065e18466f8495e6f3dae8c02a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 21:08:56.087010 24030 master.cc:537] Master@127.11.207.189:37091 shutting down...
I20260812 21:08:56.087015 24234 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:56.087117 24231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } }
I20260812 21:08:56.087591 24231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
W20260812 21:08:56.088217 23095 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 21:08:56.088846 23095 consensus_peers.cc:487] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a -> Peer f48f780f144f4656959ad5e5fc9f084b (127.11.207.189:37091): Couldn't send request to peer f48f780f144f4656959ad5e5fc9f084b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 21:08:56.089798 24030 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:56.089895 24030 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:56.089948 24030 tablet_replica.cc:330] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: stopping tablet replica
I20260812 21:08:56.102842 24030 master.cc:552] Master@127.11.207.189:37091 shutdown complete.
I20260812 21:08:56.102896 24030 master_runner.cc:305] Clearing existing system tablet
I20260812 21:08:56.103018 24030 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.103067 24030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.103641 24030 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260812 21:08:56.104092 24245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.104233 24030 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.104290 24030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.104312 24030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
uuid: "f48f780f144f4656959ad5e5fc9f084b"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "47f570351c430b8aaeae6e5361c3857a"
server_key_iv: "28faada2b145de9f1fc4af44ae9da268"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.105058 24030 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:56.106361 24030 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 21:08:56.106423 24030 log.cc:1196] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 21:08:56.106606 24030 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Deleting consensus metadata
I20260812 21:08:56.106715 24030 master_runner.cc:315] Copying system tablet from 127.11.207.190:34819
I20260812 21:08:56.107316 24030 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: tablet copy: Beginning tablet copy session from remote peer at address 127.11.207.190:34819
I20260812 21:08:56.113670 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.107682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33314)
0812 21:08:56.107796 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.107806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.107879 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.108057 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.108058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.108235 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.108296 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.108298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.109267 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.109268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.109806 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.109808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.110291 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.110292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.110345 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.110774 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.110780 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.111098 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.112739 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.112740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.112740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.112915 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.113115 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.113115 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.113115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.113161 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.113426 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.113426 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.113463 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.113507 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.113585 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":69,"threads_started":1}
I20260812 21:08:56.113709 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.107760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.11.207.190:34819)
0812 21:08:56.107816 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.107818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.107878 (+ 60us) server_negotiation.cc:407] Connection header received
0812 21:08:56.107930 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.107931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.107943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.108047 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.108394 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.109187 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.109903 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.110221 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.110294 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.111194 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.111196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.111196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.111205 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.112666 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.112986 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.112987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.112988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.113057 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.113212 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.113213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.113213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.113357 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.113443 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.113534 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.113616 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:08:56.114238 23159 tablet_copy_service.cc:140] P 286919065e18466f8495e6f3dae8c02a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f48f780f144f4656959ad5e5fc9f084b ({username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:33314)
I20260812 21:08:56.114382 23159 tablet_copy_service.cc:161] P 286919065e18466f8495e6f3dae8c02a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f48f780f144f4656959ad5e5fc9f084b at {username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:33314: session id = f48f780f144f4656959ad5e5fc9f084b-00000000000000000000000000000000
I20260812 21:08:56.115163 23159 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 21:08:56.115772 24030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 21:08:56.117051 24030 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: tablet copy: Starting download of 0 data blocks...
I20260812 21:08:56.117202 24030 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: tablet copy: Starting download of 1 WAL segments...
I20260812 21:08:56.118098 24030 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 21:08:56.118912 23159 tablet_copy_service.cc:342] P 286919065e18466f8495e6f3dae8c02a: Request end of tablet copy session f48f780f144f4656959ad5e5fc9f084b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:33314
I20260812 21:08:56.118965 23159 tablet_copy_service.cc:433] P 286919065e18466f8495e6f3dae8c02a: ending tablet copy session f48f780f144f4656959ad5e5fc9f084b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f48f780f144f4656959ad5e5fc9f084b
I20260812 21:08:56.121381 24030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:08:56.122601 24030 server_base.cc:1034] running on GCE node
W20260812 21:08:56.122581 24266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.122658 24263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.122591 24261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.122893 24030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.123029 24030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.124158 24030 hybrid_clock.cc:648] HybridClock initialized: now 1786568936124151 us; error 23 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.125492 24030 init.cc:377] Logged in from keytab as oryx/127.11.207.189@KRBTEST.COM (short username oryx)
I20260812 21:08:56.125751 24030 webserver.cc:460] Webserver started at http://127.11.207.189:35011/ using document root <none> and password file <none>
I20260812 21:08:56.125865 24030 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.125900 24030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.126595 24030 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:56.127134 24289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.127261 24030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:56.127306 24030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
uuid: "f48f780f144f4656959ad5e5fc9f084b"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "47f570351c430b8aaeae6e5361c3857a"
server_key_iv: "28faada2b145de9f1fc4af44ae9da268"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.127370 24030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.142272 24030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.143882 24030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.148284 24030 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.189:37091
I20260812 21:08:56.148438 24379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.189:37091 every 8 connection(s)
I20260812 21:08:56.148676 24030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
I20260812 21:08:56.149551 24380 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:08:56.149987 24380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Bootstrap starting.
I20260812 21:08:56.152907 24380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:08:56.153242 24380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Bootstrap complete.
I20260812 21:08:56.153505 24380 raft_consensus.cc:348] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } }
I20260812 21:08:56.153601 24380 raft_consensus.cc:729] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f48f780f144f4656959ad5e5fc9f084b, State: Initialized, Role: LEARNER
I20260812 21:08:56.153643 24380 consensus_queue.cc:260] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } }
I20260812 21:08:56.153843 24380 sys_catalog.cc:564] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:56.153860 24387 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } }
I20260812 21:08:56.154019 24387 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: LEARNER
I20260812 21:08:56.155696 24413 catalog_manager.cc:1259] Loaded cluster ID: c502e73b744e4a8da3609363b00bb5eb
I20260812 21:08:56.155781 24413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: loading cluster ID for follower catalog manager: success
I20260812 21:08:56.157249 24413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: acquiring CA information for follower catalog manager: success
I20260812 21:08:56.157640 24413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:08:56.159039 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.151091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.11.207.189:37091)
0812 21:08:56.151262 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.151264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.151271 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:56.151295 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.151296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.151304 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.151360 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.152063 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.153201 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.154213 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.154536 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.154614 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.156270 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.156272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.156272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.156279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.157835 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.158282 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.158283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.158284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.158351 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.158571 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.158571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.158572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.158672 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.158741 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.158894 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.158976 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for oryx/127.11.207.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.188.keytab.
Entry for principal oryx/127.11.207.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.188.keytab.
I20260812 21:08:56.191963 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:36971
--webserver_interface=127.11.207.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.188.keytab
--principal=oryx/127.11.207.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:56.287974 24417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:56.288178 24417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:56.288200 24417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:56.289377 24417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:56.289422 24417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:56.289434 24417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:56.289445 24417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:56.289455 24417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:56.291074 24417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.188.keytab
--principal=oryx/127.11.207.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:36971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.207.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:56.291319 24417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:56.291531 24417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:56.296525 24424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.296535 24423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.296551 24426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.296790 24417 server_base.cc:1034] running on GCE node
I20260812 21:08:56.297060 24417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.297360 24417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.298534 24417 hybrid_clock.cc:648] HybridClock initialized: now 1786568936298510 us; error 33 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.301298 24417 init.cc:377] Logged in from keytab as oryx/127.11.207.188@KRBTEST.COM (short username oryx)
I20260812 21:08:56.301729 24417 webserver.cc:460] Webserver started at http://127.11.207.188:34745/ using document root <none> and password file <none>
I20260812 21:08:56.301941 24417 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.301990 24417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.302124 24417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:56.303246 24417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/instance:
uuid: "6354a889622345c2aaafcc5ba75e0ddf"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "a73f9d5edaa83e4bb3eea89e168be8c1"
server_key_iv: "742c14f20b19d2ec63b18518f5eb298a"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.303606 24417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal/instance:
uuid: "6354a889622345c2aaafcc5ba75e0ddf"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "a73f9d5edaa83e4bb3eea89e168be8c1"
server_key_iv: "742c14f20b19d2ec63b18518f5eb298a"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.304950 24417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:56.305677 24438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.305838 24417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:56.305924 24417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
uuid: "6354a889622345c2aaafcc5ba75e0ddf"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "a73f9d5edaa83e4bb3eea89e168be8c1"
server_key_iv: "742c14f20b19d2ec63b18518f5eb298a"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.305984 24417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.319111 24417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.321388 24417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:56.321525 24417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.327148 24417 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.188:36971
I20260812 21:08:56.327198 24549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.188:36971 every 8 connection(s)
I20260812 21:08:56.327549 24417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
I20260812 21:08:56.328639 24550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:56.328917 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 24417
I20260812 21:08:56.331183 24550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.207.188@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.188@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:08:56.339591 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.332001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33322 (local address 127.11.207.190:34819)
0812 21:08:56.332077 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.332079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.332090 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:56.332117 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.332118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.332129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.332196 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.332689 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.333222 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.334146 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.334349 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.334403 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.336884 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.336886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.336886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.336894 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.338195 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.338785 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.338787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.338788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.338852 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.339051 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.339052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.339052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.339184 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.339248 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.339424 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.339504 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:08:56.339946 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.331811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33322)
0812 21:08:56.331966 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.332010 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.332070 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.332220 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.332221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.332516 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.332608 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.332611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.333359 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.333360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.334066 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.334069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.334444 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.334446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.334503 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.334976 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.334984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.336769 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.338290 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.338296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.338298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.338650 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.338918 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.338919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.338919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.338990 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.339239 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.339243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.339310 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.339401 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.339576 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":18,"thread_start_us":73,"threads_started":1}
I20260812 21:08:56.340101 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.329590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.11.207.188:36971)
0812 21:08:56.329739 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.329755 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.329781 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:08:56.330309 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.330317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.330369 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.330486 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.331234 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.332487 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.333312 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.333761 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.333838 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.336206 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.336215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.336215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.336222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.337899 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.338298 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.338301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.338301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.338437 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.338653 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.338654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.338655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.339022 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.339186 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.339491 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.339638 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260812 21:08:56.341656 24550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.188@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
I20260812 21:08:56.347905 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.342026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42406 (local address 127.11.207.189:37091)
0812 21:08:56.342073 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.342075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.342082 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:56.342107 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.342108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.342118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.342166 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.342458 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.342955 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.343600 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.343822 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.343876 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.345716 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.345718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.345718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.345725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.346817 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.347102 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.347103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.347103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.347150 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.347308 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.347308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.347308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.347402 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.347446 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.347742 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.347840 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:56.347903 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.341947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42406)
0812 21:08:56.341997 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.342008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.342047 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.342180 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.342181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.342333 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.342385 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.342387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.343021 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.343022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.343531 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.343533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.343878 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.343879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.343944 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.344309 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.344314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.345631 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.346891 (+ 1260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.346892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.346893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.347045 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.347191 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.347191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.347191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.347242 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.347502 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.347503 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.347587 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.347718 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.347828 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:08:56.348300 24550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.188@KRBTEST.COM for oryx/127.11.207.188@KRBTEST.COM
I20260812 21:08:56.353921 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.348636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.11.207.188:36971)
0812 21:08:56.348673 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.348675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.348717 (+ 42us) server_negotiation.cc:407] Connection header received
0812 21:08:56.348765 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.348766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.348775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.348824 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.349036 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.349506 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.350103 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.350310 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.350362 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.351882 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.351883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.351884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.351890 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.352977 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.353246 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.353247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.353248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.353311 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.353474 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.353475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.353475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.353572 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.353625 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.353746 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.353827 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:08:56.354001 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.348611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51506)
0812 21:08:56.348653 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.348662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.348716 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.348825 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.348825 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.348951 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.348979 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.348980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.349571 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.349572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.350040 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.350041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.350364 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.350365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.350410 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.350732 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.350736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.351813 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.353046 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.353046 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.353047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.353192 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.353363 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.353364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.353364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.353427 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.353623 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.353624 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.353661 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.353720 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.353904 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":65}
I20260812 21:08:56.355525 24550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Bootstrap starting.
I20260812 21:08:56.356639 24550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:56.357019 24550 log.cc:824] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:56.357905 24550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: No bootstrap required, opened a new log
I20260812 21:08:56.359625 24550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } }
I20260812 21:08:56.359764 24550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:56.359797 24550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6354a889622345c2aaafcc5ba75e0ddf, State: Initialized, Role: FOLLOWER
I20260812 21:08:56.359925 24550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } }
I20260812 21:08:56.360375 24550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:56.360320 24556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } } }
I20260812 21:08:56.360435 24556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:08:56.362860 24583 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:56.362926 24583 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:56.365202 24417 master_runner.cc:428] Detected that this master 6354a889622345c2aaafcc5ba75e0ddf is joining an existing cluster
I20260812 21:08:56.365253 24417 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.207.188:36971
I20260812 21:08:56.368841 24556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:56.368942 24556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } }
I20260812 21:08:56.369189 24556 leader_election.cc:288] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), f48f780f144f4656959ad5e5fc9f084b (127.11.207.189:37091)
I20260812 21:08:56.369547 24338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6354a889622345c2aaafcc5ba75e0ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f48f780f144f4656959ad5e5fc9f084b" is_pre_election: true
I20260812 21:08:56.369807 24338 raft_consensus.cc:1799] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Handling vote request from an unknown peer 6354a889622345c2aaafcc5ba75e0ddf
I20260812 21:08:56.369889 24338 raft_consensus.cc:2396] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 6354a889622345c2aaafcc5ba75e0ddf for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 21:08:56.370321 23147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6354a889622345c2aaafcc5ba75e0ddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "286919065e18466f8495e6f3dae8c02a" is_pre_election: true
I20260812 21:08:56.370679 24448 leader_election.cc:302] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6354a889622345c2aaafcc5ba75e0ddf; no voters: 286919065e18466f8495e6f3dae8c02a, f48f780f144f4656959ad5e5fc9f084b
I20260812 21:08:56.370846 24556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:56.371891 24556 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 21:08:56.372990 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.366348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.11.207.190:34819)
0812 21:08:56.366398 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.366400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.366554 (+ 154us) server_negotiation.cc:407] Connection header received
0812 21:08:56.366585 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.366586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.366598 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.366655 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.367228 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.367851 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.368982 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.369269 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.369308 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.370547 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.370548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.370549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.370556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.371874 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.372211 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.372213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.372213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.372280 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.372509 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.372510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.372510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.372625 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.372682 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.372826 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.372905 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:56.373154 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.366266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33328)
0812 21:08:56.366443 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.366454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.366514 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.366661 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.366661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.367051 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.367130 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.367133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.367925 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.367926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.368884 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.368887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.369397 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.369397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.369458 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.369977 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.369982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.370410 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.371952 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.371953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.371953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.372147 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.372334 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.372335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.372335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.372454 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.372681 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.372682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.372729 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.372797 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.372908 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":453,"threads_started":1}
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568935, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.190@KRBTEST.COM for oryx/127.11.207.188@KRBTEST.COM
I20260812 21:08:56.392488 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.375444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51530)
0812 21:08:56.375490 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.375499 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.375531 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.375826 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.375827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.375976 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.376013 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.376015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.376731 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.376732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.377241 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.377243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.377683 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.377684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.377755 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.378180 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.378185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.379503 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.391281 (+ 11778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.391283 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.391284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.391521 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.391751 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.391752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.391752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.391829 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.392139 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.392140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.392194 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.392238 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.392388 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:08:56.392565 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 23236
I20260812 21:08:56.392926 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.375507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51530 (local address 127.11.207.188:36971)
0812 21:08:56.375683 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.375685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.375692 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:56.375727 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.375728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.375737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.375805 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.376079 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.376660 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.377327 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.377613 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.377673 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.379616 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.379618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.379618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.379625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.391159 (+ 11534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.391608 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.391609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.391609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.391676 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.391902 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.391903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.391903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.392063 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.392681 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.392722 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.392807 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260812 21:08:56.398679 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.366271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42416)
0812 21:08:56.366840 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.366851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.366929 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.367051 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.367052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.367324 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.367386 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.367388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.368306 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.368307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.369110 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.369111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.369565 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.369566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.369624 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.389127 (+ 19503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.389135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.389767 (+ 632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.392181 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.392182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.392183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.392418 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.392631 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.392631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.392632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.392700 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.398201 (+ 5501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.398203 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.398283 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.398375 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.398537 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":116,"threads_started":1}
I20260812 21:08:56.399096 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.366306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.11.207.189:37091)
0812 21:08:56.366353 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.366355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.366919 (+ 564us) server_negotiation.cc:407] Connection header received
0812 21:08:56.366976 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.366977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.366987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.367028 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.367473 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.368217 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.369191 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.369454 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.369491 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.389922 (+ 20431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.389926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.389926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.389938 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.392061 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.392505 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.392507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.392507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.392568 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.397891 (+ 5323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.397894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.397894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.398097 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.398189 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.398885 (+ 696us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.398995 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:56.406908 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.366561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.11.207.188:36971)
0812 21:08:56.366742 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.366743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.372508 (+ 5765us) server_negotiation.cc:407] Connection header received
0812 21:08:56.372590 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.372590 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.372599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.372623 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.373413 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.374123 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.375391 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.375654 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.375720 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.390042 (+ 14322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.390043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.390043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.390050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.391556 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.391902 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.391903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.391903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.406041 (+ 14138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.406329 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.406330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.406331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.406471 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.406516 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.406706 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.406807 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:56.407236 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.366524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51514)
0812 21:08:56.372442 (+ 5918us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.372453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.372530 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.372672 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.372673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.372853 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.373279 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.373283 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.374339 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.374340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.375286 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.375289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.375726 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.375727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.375798 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.389498 (+ 13700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.389503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.389951 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.391628 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.391628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.391629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.391831 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.406179 (+ 14348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.406180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.406181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.406262 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.406554 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.406555 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.406603 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.407024 (+ 421us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.407128 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5899,"thread_start_us":267,"threads_started":1}
I20260812 21:08:56.408679 23122 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.207.188:36971: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf"
member_type: NON_VOTER
last_known_addr {
host: "127.11.207.188"
port: 36971
}
attrs {
promote: true
}
}
dest_uuid: "286919065e18466f8495e6f3dae8c02a"
cas_config_opid_index: 5
I20260812 21:08:56.408927 23122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } }
I20260812 21:08:56.409451 24234 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.410292 24234 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.207.188:36971
I20260812 21:08:56.410378 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.410436 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.410655 24496 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Refusing update from remote peer 286919065e18466f8495e6f3dae8c02a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 21:08:56.410908 24236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:56.410990 24417 master.cc:537] Master@127.11.207.188:36971 shutting down...
W20260812 21:08:56.412397 23095 consensus_peers.cc:487] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a -> Peer 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188:36971): Couldn't send request to peer 6354a889622345c2aaafcc5ba75e0ddf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 21:08:56.414036 24417 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:56.414130 24417 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:56.414178 24417 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: stopping tablet replica
I20260812 21:08:56.415261 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:40121
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:34819
--webserver_port=38973
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:08:56.416486 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.409366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42428)
0812 21:08:56.411761 (+ 2395us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.411774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.411812 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.413170 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.413172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.413174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:56.413229 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.413231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.414151 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.414153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.415295 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.415299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.416228 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.416229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.416291 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.416319 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.416389 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.416434 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2376}
I20260812 21:08:56.416741 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.412440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.11.207.189:37091)
0812 21:08:56.412737 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.412740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.412749 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:56.412776 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.412777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.412806 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.412876 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:56.413332 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.414055 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.415396 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.416049 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.416113 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.416609 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.416632 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.416675 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260812 21:08:56.417105 24338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Refusing update from remote peer 286919065e18466f8495e6f3dae8c02a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 21:08:56.417382 24236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 21:08:56.417980 24338 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.419260 24598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.419405 24387 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 286919065e18466f8495e6f3dae8c02a. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: NON_VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: true } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.419529 24387 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: LEARNER
I20260812 21:08:56.419426 24598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: LEARNER
I20260812 21:08:56.419719 24234 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: attempting to promote NON_VOTER f48f780f144f4656959ad5e5fc9f084b to VOTER
I20260812 21:08:56.419957 24234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } }
I20260812 21:08:56.420449 24338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 LEARNER]: Refusing update from remote peer 286919065e18466f8495e6f3dae8c02a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 21:08:56.420584 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.420410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51540)
0812 21:08:56.420453 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.420493 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:36971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 21:08:56.420650 24236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 21:08:56.420982 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.420880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51554)
0812 21:08:56.420928 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.420958 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:36971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 21:08:56.421314 23095 consensus_peers.cc:487] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a -> Peer 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188:36971): Couldn't send request to peer 6354a889622345c2aaafcc5ba75e0ddf. Status: Network error: Client connection negotiation failed: client connection to 127.11.207.188:36971: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 21:08:56.421479 24231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f48f780f144f4656959ad5e5fc9f084b (127.11.207.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.421633 24338 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f48f780f144f4656959ad5e5fc9f084b (127.11.207.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.422442 24231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.422529 24231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.422458 24598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.422645 24598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:56.422705 24231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.422782 24231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.428167 24417 master.cc:552] Master@127.11.207.188:36971 shutdown complete.
I20260812 21:08:56.428226 24417 master_runner.cc:305] Clearing existing system tablet
I20260812 21:08:56.428391 24417 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.428440 24417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.429169 24417 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:08:56.429688 24608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.429807 24417 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.429857 24417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.429878 24417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
uuid: "6354a889622345c2aaafcc5ba75e0ddf"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "a73f9d5edaa83e4bb3eea89e168be8c1"
server_key_iv: "742c14f20b19d2ec63b18518f5eb298a"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.430692 24417 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:56.432159 24417 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 21:08:56.432236 24417 log.cc:1196] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 21:08:56.432461 24417 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Deleting consensus metadata
I20260812 21:08:56.432585 24417 master_runner.cc:315] Copying system tablet from 127.11.207.190:34819
I20260812 21:08:56.433228 24417 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: tablet copy: Beginning tablet copy session from remote peer at address 127.11.207.190:34819
I20260812 21:08:56.440727 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.433621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.11.207.190:34819)
0812 21:08:56.433673 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.433675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.433734 (+ 59us) server_negotiation.cc:407] Connection header received
0812 21:08:56.433798 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.433799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.433809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.433886 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.434286 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.435081 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.436152 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.436497 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.436566 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.437641 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.437643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.437644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.437652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.439233 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.439680 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.439682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.439683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.439763 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.439987 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.439988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.439989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.440191 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.440240 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.440545 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.440631 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:56.441298 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.433556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33332)
0812 21:08:56.433677 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.433688 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.433745 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.433899 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.433900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.434108 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.434178 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.434181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.435175 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.435177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.436060 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.436062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.436568 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.436569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.436640 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.437125 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.437131 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.437546 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.439368 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.439369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.439370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.439601 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.439833 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.439833 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.439834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.439901 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.440296 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.440297 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.440426 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.441068 (+ 642us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.441191 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260812 21:08:56.441772 23159 tablet_copy_service.cc:140] P 286919065e18466f8495e6f3dae8c02a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6354a889622345c2aaafcc5ba75e0ddf ({username='oryx', principal='oryx/127.11.207.188@KRBTEST.COM'} at 127.0.0.1:33332)
I20260812 21:08:56.441838 23159 tablet_copy_service.cc:161] P 286919065e18466f8495e6f3dae8c02a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6354a889622345c2aaafcc5ba75e0ddf at {username='oryx', principal='oryx/127.11.207.188@KRBTEST.COM'} at 127.0.0.1:33332: session id = 6354a889622345c2aaafcc5ba75e0ddf-00000000000000000000000000000000
I20260812 21:08:56.442433 23159 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 21:08:56.442992 24417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 21:08:56.444443 24417 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: tablet copy: Starting download of 0 data blocks...
I20260812 21:08:56.444676 24417 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: tablet copy: Starting download of 1 WAL segments...
I20260812 21:08:56.445616 24417 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 21:08:56.446672 23159 tablet_copy_service.cc:342] P 286919065e18466f8495e6f3dae8c02a: Request end of tablet copy session 6354a889622345c2aaafcc5ba75e0ddf-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.207.188@KRBTEST.COM'} at 127.0.0.1:33332
I20260812 21:08:56.446728 23159 tablet_copy_service.cc:433] P 286919065e18466f8495e6f3dae8c02a: ending tablet copy session 6354a889622345c2aaafcc5ba75e0ddf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6354a889622345c2aaafcc5ba75e0ddf
I20260812 21:08:56.449847 24417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:56.451066 24619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.453264 24626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.453440 24629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.453642 24417 server_base.cc:1034] running on GCE node
I20260812 21:08:56.453850 24417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.454015 24417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.455125 24417 hybrid_clock.cc:648] HybridClock initialized: now 1786568936455110 us; error 37 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.456768 24417 init.cc:377] Logged in from keytab as oryx/127.11.207.188@KRBTEST.COM (short username oryx)
I20260812 21:08:56.457108 24417 webserver.cc:460] Webserver started at http://127.11.207.188:33513/ using document root <none> and password file <none>
I20260812 21:08:56.457285 24417 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.457370 24417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.458137 24417 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.458770 24639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.458913 24417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.458950 24417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
uuid: "6354a889622345c2aaafcc5ba75e0ddf"
format_stamp: "Formatted at 2026-08-12 21:08:56 on dist-test-slave-lkx2"
server_key: "a73f9d5edaa83e4bb3eea89e168be8c1"
server_key_iv: "742c14f20b19d2ec63b18518f5eb298a"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.459004 24417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.485080 24417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.486991 24417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.490556 24417 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.188:36971
I20260812 21:08:56.490680 24736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.188:36971 every 8 connection(s)
I20260812 21:08:56.491025 24417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
I20260812 21:08:56.492072 24738 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:08:56.492645 24738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Bootstrap starting.
I20260812 21:08:56.495630 24738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:08:56.495843 24738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: Bootstrap complete.
I20260812 21:08:56.496047 24738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } }
I20260812 21:08:56.496141 24738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6354a889622345c2aaafcc5ba75e0ddf, State: Initialized, Role: LEARNER
I20260812 21:08:56.496201 24738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } }
I20260812 21:08:56.496444 24738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:56.496404 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.496611 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: This master's current role is: LEARNER
I20260812 21:08:56.497985 24777 catalog_manager.cc:1259] Loaded cluster ID: c502e73b744e4a8da3609363b00bb5eb
I20260812 21:08:56.498075 24777 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: loading cluster ID for follower catalog manager: success
I20260812 21:08:56.499472 24777 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: acquiring CA information for follower catalog manager: success
I20260812 21:08:56.499770 24777 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 21:08:56.523187 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:56.523406 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:56.523429 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:56.524639 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:56.524682 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:56.524708 24595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:08:56.526309 24595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:40121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=38973
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:56.526553 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:56.526777 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:56.531944 24787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.532065 24595 server_base.cc:1034] running on GCE node
W20260812 21:08:56.531976 24797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.531949 24788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.532374 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.532624 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.533774 24595 hybrid_clock.cc:648] HybridClock initialized: now 1786568936533759 us; error 33 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.536568 24595 init.cc:377] Logged in from keytab as oryx/127.11.207.129@KRBTEST.COM (short username oryx)
I20260812 21:08:56.537050 24595 webserver.cc:460] Webserver started at http://127.11.207.129:38973/ using document root <none> and password file <none>
I20260812 21:08:56.537257 24595 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.537308 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.538683 24595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:56.539382 24805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.539583 24595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:56.539664 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "b3078d23e0d34fe5b5011d6c50e1da9c"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "44c5bb0f418f1049627698b4079cf408"
server_key_iv: "f628fc17f9efff527c56545426a4a15a"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.540055 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.550001 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.551942 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:56.552067 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.552270 24595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:56.552609 24595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:56.552645 24595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.552666 24595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:56.552685 24595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.560241 24595 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:40121
I20260812 21:08:56.560329 25012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:40121 every 8 connection(s)
I20260812 21:08:56.560686 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:08:56.561777 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 24595
I20260812 21:08:56.561951 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 23483
I20260812 21:08:56.582791 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:35089
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:34819
--webserver_port=33149
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.188@KRBTEST.COM
I20260812 21:08:56.588172 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.561861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:46219 (local address 127.11.207.189:37091)
0812 21:08:56.561909 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.561911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.577300 (+ 15389us) server_negotiation.cc:407] Connection header received
0812 21:08:56.578531 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.578532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.578547 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.578580 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.579735 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.580548 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582118 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.582418 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582459 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.585277 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.585279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.585279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.585288 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.586759 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587244 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.587245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.587245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.587307 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587537 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.587538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.587538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.587663 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.587720 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.587995 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.588077 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:56.588505 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.561785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.11.207.129:46219)
0812 21:08:56.577217 (+ 15432us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.577226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.578408 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.578790 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.578795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.579278 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.579618 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.579624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.580730 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.580730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.582013 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.582015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.582886 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.582946 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.583470 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.583474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.585171 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.586845 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.586853 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.586856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.587143 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.587368 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.587369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.587479 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.587733 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.587740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.587874 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.587968 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.588134 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15416,"mutex_wait_us":101,"thread_start_us":14988,"threads_started":1}
I20260812 21:08:56.588506 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.561455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:55345)
0812 21:08:56.576973 (+ 15518us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.576987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.578409 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.578790 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.578795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.579294 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.579618 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.579624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.580693 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.580694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.582012 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.582015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.582540 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.582644 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.583349 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.583355 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.585541 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.587261 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.587262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.587472 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.587773 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.587774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.587845 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.588136 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.588137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.588182 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.588242 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.588366 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15482,"mutex_wait_us":46,"thread_start_us":151,"threads_started":1}
I20260812 21:08:56.589336 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.561457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:55345 (local address 127.11.207.190:34819)
0812 21:08:56.561511 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.561513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.577087 (+ 15574us) server_negotiation.cc:407] Connection header received
0812 21:08:56.578532 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.578533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.578547 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.578579 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.579735 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.580556 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582118 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.582426 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582461 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.585638 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.585639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.585640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.585648 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.587198 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587636 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.587638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.587638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.587702 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.587912 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.587913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.587914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.588051 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.588109 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.588764 (+ 655us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.588867 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:08:56.589699 25018 heartbeater.cc:347] Connected to a master server at 127.11.207.189:37091
I20260812 21:08:56.589795 25018 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.589928 25014 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34819
I20260812 21:08:56.589989 25014 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.590077 25014 heartbeater.cc:511] Master 127.11.207.190:34819 requested a full tablet report, sending...
I20260812 21:08:56.589994 25018 heartbeater.cc:511] Master 127.11.207.189:37091 requested a full tablet report, sending...
I20260812 21:08:56.590605 23122 ts_manager.cc:194] Re-registered known tserver with Master: b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121)
I20260812 21:08:56.590672 24322 ts_manager.cc:194] Registered new tserver with Master: b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121)
I20260812 21:08:56.591109 23122 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:55345
I20260812 21:08:56.591877 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.561481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:55897 (local address 127.11.207.188:36971)
0812 21:08:56.561615 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.561617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.577087 (+ 15470us) server_negotiation.cc:407] Connection header received
0812 21:08:56.578539 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.578541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.578557 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.578599 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.579743 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.580740 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582112 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.582392 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582432 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.588653 (+ 6221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.588654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.588655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.588664 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.590651 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.591014 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.591016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.591016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.591080 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.591371 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.591372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.591372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.591482 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.591540 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.591680 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.591780 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260812 21:08:56.591887 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.561456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.11.207.129:55897)
0812 21:08:56.576894 (+ 15438us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.576970 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.578401 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.578790 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.578795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.579292 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.579618 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.579624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.580838 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.580839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.582012 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.582015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.582527 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.582527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.582639 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.583282 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.583293 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.588573 (+ 5280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.590743 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.590744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.590744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.590946 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.591134 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.591134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.591135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.591194 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.591545 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.591546 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.591588 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.591650 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.591786 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15382,"mutex_wait_us":64,"thread_start_us":115,"threads_started":1}
I20260812 21:08:56.592306 25013 heartbeater.cc:347] Connected to a master server at 127.11.207.188:36971
I20260812 21:08:56.592411 25013 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.592486 25013 heartbeater.cc:511] Master 127.11.207.188:36971 requested a full tablet report, sending...
I20260812 21:08:56.592846 24671 ts_manager.cc:194] Registered new tserver with Master: b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121)
W20260812 21:08:56.671092 25027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:56.671303 25027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:56.671344 25027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:56.672549 25027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:56.672591 25027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:56.672618 25027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:08:56.674160 25027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:35089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=33149
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:56.674399 25027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:56.674634 25027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:56.679621 25043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.679641 25046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.679628 25041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.679663 25027 server_base.cc:1034] running on GCE node
I20260812 21:08:56.680070 25027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.680305 25027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.681492 25027 hybrid_clock.cc:648] HybridClock initialized: now 1786568936681483 us; error 31 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.683948 25027 init.cc:377] Logged in from keytab as oryx/127.11.207.130@KRBTEST.COM (short username oryx)
I20260812 21:08:56.684437 25027 webserver.cc:460] Webserver started at http://127.11.207.130:33149/ using document root <none> and password file <none>
I20260812 21:08:56.684650 25027 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.684717 25027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.686043 25027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:08:56.686590 25060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.686774 25027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.686882 25027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "bdbffa9bc5594328be86882ff6e48ffa"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "3944b29f81c2c198f1ebe68daacbb344"
server_key_iv: "e8fadb9bd9f9ca80e9f935465ffaa200"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.687229 25027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.721575 25027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.723851 25027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:56.724002 25027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.724246 25027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:56.724583 25027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:56.724619 25027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.724649 25027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:56.724670 25027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.732013 25027 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:35089
I20260812 21:08:56.732061 25283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:35089 every 8 connection(s)
I20260812 21:08:56.732445 25027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:08:56.739733 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 25027
I20260812 21:08:56.739884 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 23761
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.188@KRBTEST.COM
I20260812 21:08:56.750478 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.733684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:34611 (local address 127.11.207.188:36971)
0812 21:08:56.733733 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.733735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.733886 (+ 151us) server_negotiation.cc:407] Connection header received
0812 21:08:56.734713 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.734715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.734731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.734761 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.735862 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.736508 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.737657 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.737956 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.737995 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.743134 (+ 5139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.743136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.743136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.743144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.748328 (+ 5184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.749067 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.749069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.749069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.749144 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.749510 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.749511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.749511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.749677 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.749726 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.750236 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.750366 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:08:56.751449 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.733695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.11.207.130:34611)
0812 21:08:56.733822 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.733832 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.734608 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.734942 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.734946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.735481 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.735751 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.735757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.736638 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.736639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.737558 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.737561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.738077 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.738078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.738198 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.738903 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.738908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.743052 (+ 4144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.748527 (+ 5475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.748544 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.748548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.748961 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.749229 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.749230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.749230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.749419 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.749780 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.749791 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.750014 (+ 223us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.750892 (+ 878us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.751149 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":23,"thread_start_us":76,"threads_started":1}
I20260812 21:08:56.752271 25284 heartbeater.cc:347] Connected to a master server at 127.11.207.188:36971
I20260812 21:08:56.752390 25284 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.752570 25284 heartbeater.cc:511] Master 127.11.207.188:36971 requested a full tablet report, sending...
I20260812 21:08:56.753015 24671 ts_manager.cc:194] Registered new tserver with Master: bdbffa9bc5594328be86882ff6e48ffa (127.11.207.130:35089)
I20260812 21:08:56.760093 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.733192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.11.207.130:60687)
0812 21:08:56.733377 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.733433 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.734596 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.734942 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.734947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.735439 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.735751 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.735757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.736744 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.736745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.737678 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.737680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.738136 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.738137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.738246 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.738860 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.738865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.741597 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.758894 (+ 17297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.758895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.758895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.759155 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.759426 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.759427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.759427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.759478 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.759746 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.759747 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.759794 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.759876 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.759983 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260812 21:08:56.760115 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.733211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:60687 (local address 127.11.207.189:37091)
0812 21:08:56.733259 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.733261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.733566 (+ 305us) server_negotiation.cc:407] Connection header received
0812 21:08:56.734720 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.734721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.734735 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.734766 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.735863 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.736655 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.737753 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.738034 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.738087 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.741672 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.741674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.741674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.741680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.758802 (+ 17122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.759241 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.759242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.759242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.759344 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.759534 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.759535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.759536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.759660 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.759719 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.759932 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.760025 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:56.760576 25289 heartbeater.cc:347] Connected to a master server at 127.11.207.189:37091
I20260812 21:08:56.760632 25289 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.760703 25289 heartbeater.cc:511] Master 127.11.207.189:37091 requested a full tablet report, sending...
I20260812 21:08:56.761070 24322 ts_manager.cc:194] Registered new tserver with Master: bdbffa9bc5594328be86882ff6e48ffa (127.11.207.130:35089)
I20260812 21:08:56.761194 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.733193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:35851)
0812 21:08:56.733378 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.733432 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.734608 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.734942 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.734947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.735439 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.735751 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.735757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.736747 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.736747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.737702 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.737703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.738067 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.738067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.738195 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.738806 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.738822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.741386 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.759726 (+ 18340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.759727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.759727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.759925 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.760448 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.760449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.760450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.760513 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.760832 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.760832 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.760878 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.760966 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.761067 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":50,"thread_start_us":107,"threads_started":1}
I20260812 21:08:56.761200 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.733174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:35851 (local address 127.11.207.190:34819)
0812 21:08:56.733256 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.733258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.733563 (+ 305us) server_negotiation.cc:407] Connection header received
0812 21:08:56.734729 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.734731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.734743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.734773 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.735864 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.736652 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.737802 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.737994 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.738049 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.757903 (+ 19854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.757905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.757906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.757916 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.759635 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.760013 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.760014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.760014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.760074 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.760586 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.760587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.760587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.760714 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.760748 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.761019 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.761109 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:08:56.761683 25286 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34819
I20260812 21:08:56.761736 25286 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.761791 25286 heartbeater.cc:511] Master 127.11.207.190:34819 requested a full tablet report, sending...
I20260812 21:08:56.762041 23122 ts_manager.cc:194] Re-registered known tserver with Master: bdbffa9bc5594328be86882ff6e48ffa (127.11.207.130:35089)
I20260812 21:08:56.762511 23122 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:35851
I20260812 21:08:56.764214 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:42515
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:34819
--webserver_port=37295
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:56.849491 25299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:56.849730 25299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:56.849756 25299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:56.850901 25299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:56.850946 25299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:56.850975 25299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:08:56.852555 25299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:42515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=37295
--tserver_master_addrs=127.11.207.190:34819,127.11.207.189:37091,127.11.207.188:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:56.852770 25299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:56.852981 25299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:56.858453 25312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.858395 25315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:56.858417 25309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:56.858439 25299 server_base.cc:1034] running on GCE node
I20260812 21:08:56.858870 25299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:56.859114 25299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:56.860247 25299 hybrid_clock.cc:648] HybridClock initialized: now 1786568936860210 us; error 53 us; skew 500 ppm
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:56.862803 25299 init.cc:377] Logged in from keytab as oryx/127.11.207.131@KRBTEST.COM (short username oryx)
I20260812 21:08:56.863263 25299 webserver.cc:460] Webserver started at http://127.11.207.131:37295/ using document root <none> and password file <none>
I20260812 21:08:56.863472 25299 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:56.863533 25299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:56.864964 25299 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:56.865525 25329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.865669 25299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:56.865756 25299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "6fb3fbaa178a46ae846877d15304b001"
format_stamp: "Formatted at 2026-08-12 21:08:55 on dist-test-slave-lkx2"
server_key: "868d9ff197cb13beb0589ed31f7cafdc"
server_key_iv: "00ac077fdcf7c65c280c5b03c7278f79"
server_key_version: "encryptionkey@0"
I20260812 21:08:56.866106 25299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:56.880398 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.877782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.11.207.188:36971)
0812 21:08:56.877850 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.877852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.877862 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:56.877892 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.877893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.877918 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.877976 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:56.878102 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.878644 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.879555 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.880033 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.880104 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.880201 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.880247 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.880287 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:08:56.880398 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.877714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51566)
0812 21:08:56.877766 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.877778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.877834 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.877988 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.877989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.877990 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:56.878046 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.878048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.878732 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.878733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.879467 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.879469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.880117 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.880118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.880183 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.880187 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.880240 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.880290 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:08:56.880753 24693 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 21:08:56.880959 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 286919065e18466f8495e6f3dae8c02a. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.881083 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: This master's current role is: LEARNER
I20260812 21:08:56.883174 25299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:56.884989 25299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:56.885102 25299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:56.885295 25299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:08:56.885599 25299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:08:56.885633 25299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.885655 25299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:08:56.885679 25299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:56.891733 25299 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:42515
I20260812 21:08:56.891786 25549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:42515 every 8 connection(s)
I20260812 21:08:56.892158 25299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:08:56.899510 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 25299
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.189@KRBTEST.COM
Aug 12 21:08:56 dist-test-slave-lkx2 krb5kdc[23048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568936, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.188@KRBTEST.COM
I20260812 21:08:56.904196 24382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.893445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:47221 (local address 127.11.207.189:37091)
0812 21:08:56.893496 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.893499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.893594 (+ 95us) server_negotiation.cc:407] Connection header received
0812 21:08:56.894589 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.894590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.894602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.894633 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.895443 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.896207 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897401 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.897678 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897744 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.901204 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.901206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.901206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.901212 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.902576 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903271 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.903272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.903273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.903331 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903521 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.903522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.903522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.903643 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.903682 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.904017 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.904102 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:08:56.904237 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.893070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37895 (local address 127.11.207.188:36971)
0812 21:08:56.893470 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.893472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.893560 (+ 88us) server_negotiation.cc:407] Connection header received
0812 21:08:56.894557 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.894559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.894576 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.894629 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.895415 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.896207 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897462 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.897728 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897792 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.901473 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.901474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.901474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.901483 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.903039 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903397 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.903398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.903398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.903459 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903647 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.903647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.903648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.903794 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.903855 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.904063 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.904146 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370}
I20260812 21:08:56.904417 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.893044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:51285)
0812 21:08:56.893226 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.893278 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.894481 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.894716 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.894717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.895142 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.895343 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.895344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.896119 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.896120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.897491 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.897493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.897916 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.897981 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.898435 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.898446 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.900070 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.902675 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.902676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.902677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.903037 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.903254 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.903255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.903400 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.903685 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.903693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.903863 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.903973 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.904147 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":17,"thread_start_us":106,"threads_started":1}
I20260812 21:08:56.904517 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.893359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.11.207.131:47221)
0812 21:08:56.893521 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.893531 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.894464 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.894706 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.894712 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.895139 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.895325 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.895333 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.896282 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.896282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.897307 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.897310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.897764 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.897917 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.898689 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.898693 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.901125 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.902649 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.902657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.902660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.903196 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.903402 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.903402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.903454 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.903727 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.903728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.903863 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.904310 (+ 447us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.904431 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":5,"thread_start_us":67,"threads_started":1}
I20260812 21:08:56.904701 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.893026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:51285 (local address 127.11.207.190:34819)
0812 21:08:56.893078 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.893080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.893374 (+ 294us) server_negotiation.cc:407] Connection header received
0812 21:08:56.894587 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.894589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.894602 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.894633 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.895437 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.896005 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897565 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.897850 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897907 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.900157 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.900166 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.900166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.900175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.902584 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903120 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.903121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.903121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.903185 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903468 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.903469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.903469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.903601 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.903658 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.904535 (+ 877us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.904616 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:56.904994 25561 heartbeater.cc:347] Connected to a master server at 127.11.207.189:37091
I20260812 21:08:56.905061 25555 heartbeater.cc:347] Connected to a master server at 127.11.207.190:34819
I20260812 21:08:56.905086 25561 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.905114 25555 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.905282 25561 heartbeater.cc:511] Master 127.11.207.189:37091 requested a full tablet report, sending...
I20260812 21:08:56.905279 25555 heartbeater.cc:511] Master 127.11.207.190:34819 requested a full tablet report, sending...
I20260812 21:08:56.905519 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.893044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.11.207.131:37895)
0812 21:08:56.893511 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.893519 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.894466 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.894703 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.894707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.895102 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:56.895324 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.895329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.896409 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.896410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.897286 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.897289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.897848 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.897849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.897916 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.898616 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:56.898620 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:56.901342 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:56.903121 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.903123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.903328 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.903523 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:56.903524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:56.903524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:56.903582 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:56.903880 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:56.903881 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:56.903925 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.903985 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.905326 (+ 1341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":1211,"thread_start_us":101,"threads_started":1}
I20260812 21:08:56.905761 24322 ts_manager.cc:194] Registered new tserver with Master: 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515)
I20260812 21:08:56.905881 25552 heartbeater.cc:347] Connected to a master server at 127.11.207.188:36971
I20260812 21:08:56.905918 25552 heartbeater.cc:464] Registering TS with master...
I20260812 21:08:56.905970 25552 heartbeater.cc:511] Master 127.11.207.188:36971 requested a full tablet report, sending...
I20260812 21:08:56.905959 23122 ts_manager.cc:194] Re-registered known tserver with Master: 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515)
I20260812 21:08:56.906216 24671 ts_manager.cc:194] Registered new tserver with Master: 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515)
I20260812 21:08:56.906401 23122 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:51285
I20260812 21:08:56.912124 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.900997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.11.207.189:37091)
0812 21:08:56.901122 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.901124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.902942 (+ 1818us) server_negotiation.cc:407] Connection header received
0812 21:08:56.903027 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.903028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.903043 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.903101 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.905623 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.906353 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.907340 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.907622 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.907697 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.909062 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.909064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.909064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.909072 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.910838 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.911217 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.911218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.911219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.911287 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.911513 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.911514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.911515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.911651 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.911726 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.911946 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.912028 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260812 21:08:56.912734 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.901189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33346 (local address 127.11.207.190:34819)
0812 21:08:56.901355 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.901357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.903258 (+ 1901us) server_negotiation.cc:407] Connection header received
0812 21:08:56.903283 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.903284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.903296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.903353 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.904087 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.907280 (+ 3193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.908102 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.908314 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.908399 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.909845 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.909847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.909847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.909855 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.911330 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.911661 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.911662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.911662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.911717 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.911900 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.911900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.911901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.912021 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.912108 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.912564 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.912647 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260812 21:08:56.913406 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.901472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51574 (local address 127.11.207.188:36971)
0812 21:08:56.901567 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.901569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.905628 (+ 4059us) server_negotiation.cc:407] Connection header received
0812 21:08:56.905652 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.905652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.905665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.905719 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:56.906389 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.907131 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.908140 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.908456 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.908522 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.909990 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:56.909992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:56.909992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:56.909999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:56.911485 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.911883 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.911884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.911884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.911951 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:56.912186 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:56.912187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:56.912187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:56.912375 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:56.912478 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.912763 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.913231 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":55,"threads_started":1}
I20260812 21:08:56.914659 23122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:08:56.916685 23122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:08:56.917412 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: NON_VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: true } } }
I20260812 21:08:56.917510 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.918793 24234 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: attempting to promote NON_VOTER 6354a889622345c2aaafcc5ba75e0ddf to VOTER
I20260812 21:08:56.918917 24234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:08:56.919557 24338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Refusing update from remote peer 286919065e18466f8495e6f3dae8c02a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 21:08:56.919569 24693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 LEARNER]: Refusing update from remote peer 286919065e18466f8495e6f3dae8c02a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 21:08:56.919776 24600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 21:08:56.920228 24600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 21:08:56.920429 24600 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.920574 24338 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.921325 24693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.921449 24600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.921530 24600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.921579 24231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.921630 24231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.921720 24598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.921797 24598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:56.921499 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.921917 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:56.922634 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "286919065e18466f8495e6f3dae8c02a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:08:56.922744 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:56.926656 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.922491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40121 (local address 127.0.0.1:59166)
0812 21:08:56.922624 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.922643 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.922696 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.923327 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.923328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.923330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:56.923384 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.923386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.924662 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.924664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.925531 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.925534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.926436 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.926438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.926500 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.926506 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.926534 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.926579 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260812 21:08:56.926837 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.922183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:42515 (local address 127.0.0.1:51632)
0812 21:08:56.922231 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.922243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.922285 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.923608 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.923609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.923611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:56.923651 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.923652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.924508 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.924509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.925451 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.925454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.926585 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.926586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.926669 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.926673 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.926720 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.926763 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":51}
I20260812 21:08:56.926924 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.922398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.11.207.131:42515)
0812 21:08:56.923462 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.923466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.923477 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:56.923517 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.923519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.923536 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.923613 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:56.923707 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.924433 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.925561 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.926499 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.926680 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.926755 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.926788 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.926840 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":105,"threads_started":1}
I20260812 21:08:56.926944 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.922283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:35089 (local address 127.0.0.1:33076)
0812 21:08:56.922407 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:56.922419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:56.922470 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:56.923413 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:56.923415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:56.923417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:56.923469 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.923471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.925052 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.925054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.925913 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:56.925915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.926750 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.926751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:56.926807 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.926811 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:08:56.926853 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 21:08:56.926887 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260812 21:08:56.927361 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.922910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.11.207.130:35089)
0812 21:08:56.923155 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.923161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.923173 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:08:56.923219 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.923222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.923240 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.923327 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:56.923855 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.924965 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.925996 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.926555 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.926666 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.927101 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.927131 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.927183 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":88,"threads_started":1}
I20260812 21:08:56.927456 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:56.923018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.11.207.129:40121)
0812 21:08:56.923171 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:56.923176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:56.923187 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:56.923227 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:56.923229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:56.923242 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:56.923326 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:56.923438 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.924564 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.925628 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:56.926338 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:56.927254 (+ 916us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:56.927328 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:56.927351 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 21:08:56.927385 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260812 21:08:56.927783 25425 tablet_service.cc:1467] Processing CreateTablet for tablet 7696a7b22abd43e9a2c94a989abb92d8 (DEFAULT_TABLE table=test-table [id=1b360b96499f479ab604de5e65c2384f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:56.927983 25155 tablet_service.cc:1467] Processing CreateTablet for tablet 7696a7b22abd43e9a2c94a989abb92d8 (DEFAULT_TABLE table=test-table [id=1b360b96499f479ab604de5e65c2384f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:56.928278 25425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7696a7b22abd43e9a2c94a989abb92d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:56.928300 25155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7696a7b22abd43e9a2c94a989abb92d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:56.932754 25607 tablet_bootstrap.cc:492] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Bootstrap starting.
I20260812 21:08:56.932823 24889 tablet_service.cc:1467] Processing CreateTablet for tablet 7696a7b22abd43e9a2c94a989abb92d8 (DEFAULT_TABLE table=test-table [id=1b360b96499f479ab604de5e65c2384f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:08:56.933223 24889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7696a7b22abd43e9a2c94a989abb92d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:56.933447 25607 tablet_bootstrap.cc:654] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:56.933790 25607 log.cc:824] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:56.934617 25607 tablet_bootstrap.cc:492] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: No bootstrap required, opened a new log
I20260812 21:08:56.934684 25607 ts_tablet_manager.cc:1397] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:08:56.936504 25610 tablet_bootstrap.cc:492] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Bootstrap starting.
I20260812 21:08:56.937009 25607 raft_consensus.cc:348] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:56.937187 25607 raft_consensus.cc:374] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:56.937229 25607 raft_consensus.cc:729] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbffa9bc5594328be86882ff6e48ffa, State: Initialized, Role: FOLLOWER
I20260812 21:08:56.937161 25611 tablet_bootstrap.cc:492] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Bootstrap starting.
I20260812 21:08:56.937372 25610 tablet_bootstrap.cc:654] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:56.937374 25607 consensus_queue.cc:260] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:56.937644 25607 ts_tablet_manager.cc:1428] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:08:56.937740 25286 heartbeater.cc:503] Master 127.11.207.190:34819 was elected leader, sending a full tablet report...
I20260812 21:08:56.937829 25610 log.cc:824] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:56.938025 25611 tablet_bootstrap.cc:654] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:56.938534 25611 log.cc:824] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:56.938572 25610 tablet_bootstrap.cc:492] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: No bootstrap required, opened a new log
I20260812 21:08:56.938653 25610 ts_tablet_manager.cc:1397] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:08:56.940745 25610 raft_consensus.cc:348] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:56.940884 25610 raft_consensus.cc:374] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:56.940922 25610 raft_consensus.cc:729] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3078d23e0d34fe5b5011d6c50e1da9c, State: Initialized, Role: FOLLOWER
I20260812 21:08:56.941036 25611 tablet_bootstrap.cc:492] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: No bootstrap required, opened a new log
I20260812 21:08:56.941044 25610 consensus_queue.cc:260] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:56.941097 25611 ts_tablet_manager.cc:1397] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 21:08:56.941373 25610 ts_tablet_manager.cc:1428] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:08:56.941404 25014 heartbeater.cc:503] Master 127.11.207.190:34819 was elected leader, sending a full tablet report...
I20260812 21:08:56.943735 25611 raft_consensus.cc:348] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:56.943873 25611 raft_consensus.cc:374] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:56.943894 25611 raft_consensus.cc:729] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb3fbaa178a46ae846877d15304b001, State: Initialized, Role: FOLLOWER
I20260812 21:08:56.943967 25611 consensus_queue.cc:260] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:56.944165 25611 ts_tablet_manager.cc:1428] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:08:56.944283 25555 heartbeater.cc:503] Master 127.11.207.190:34819 was elected leader, sending a full tablet report...
W20260812 21:08:56.983367 25294 tablet.cc:2369] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:57.041970 25612 raft_consensus.cc:482] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:57.042102 25612 raft_consensus.cc:504] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:57.042542 25612 leader_election.cc:288] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515), b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121)
I20260812 21:08:57.045430 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.042795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:49307 (local address 127.11.207.129:40121)
0812 21:08:57.042846 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:57.042848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:57.042859 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:57.042934 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:57.042935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:57.042945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:57.042991 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:57.043223 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.043688 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.044757 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.045111 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.045174 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.045183 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:57.045326 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:08:57.045361 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:57.045444 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.042703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:42515 (local address 127.11.207.130:53377)
0812 21:08:57.042782 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:57.042793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:57.042845 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:57.043017 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:57.043018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:57.043019 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:57.043108 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:57.043110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.043750 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.043751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:57.044668 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:57.044671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.045187 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.045188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:57.045257 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.045259 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:08:57.045305 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:08:57.045346 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":27}
I20260812 21:08:57.045430 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.042782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:53377 (local address 127.11.207.131:42515)
0812 21:08:57.042884 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:57.042886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:57.042896 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:57.042944 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:57.042945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:57.042952 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:57.043004 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:57.043173 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.043674 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.044752 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.045112 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.045180 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.045189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:57.045318 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 21:08:57.045356 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 21:08:57.045455 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.042732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40121 (local address 127.11.207.130:49307)
0812 21:08:57.042811 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:57.042821 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:57.042862 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:57.043011 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:57.043012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:57.043014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:57.043149 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:57.043151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.043802 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.043802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:57.044670 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:57.044673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.045184 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.045185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:57.045255 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.045258 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:08:57.045304 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:08:57.045346 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 21:08:57.045852 25458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7696a7b22abd43e9a2c94a989abb92d8" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb3fbaa178a46ae846877d15304b001" is_pre_election: true
I20260812 21:08:57.045987 24930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7696a7b22abd43e9a2c94a989abb92d8" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" is_pre_election: true
I20260812 21:08:57.046039 25458 raft_consensus.cc:2454] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 0.
I20260812 21:08:57.046131 24930 raft_consensus.cc:2454] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 0.
I20260812 21:08:57.046310 25064 leader_election.cc:302] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb3fbaa178a46ae846877d15304b001, bdbffa9bc5594328be86882ff6e48ffa; no voters:
I20260812 21:08:57.046458 25612 raft_consensus.cc:2781] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:08:57.046520 25612 raft_consensus.cc:482] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:08:57.046545 25612 raft_consensus.cc:3037] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:57.047461 25612 raft_consensus.cc:504] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:57.047598 25612 leader_election.cc:288] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 election: Requested vote from peers 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515), b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121)
I20260812 21:08:57.047755 25458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7696a7b22abd43e9a2c94a989abb92d8" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb3fbaa178a46ae846877d15304b001"
I20260812 21:08:57.047760 24930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7696a7b22abd43e9a2c94a989abb92d8" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c"
I20260812 21:08:57.047842 25458 raft_consensus.cc:3037] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:57.047849 24930 raft_consensus.cc:3037] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:57.048882 25458 raft_consensus.cc:2454] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 1.
I20260812 21:08:57.048907 24930 raft_consensus.cc:2454] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 1.
I20260812 21:08:57.049069 25063 leader_election.cc:302] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3078d23e0d34fe5b5011d6c50e1da9c, bdbffa9bc5594328be86882ff6e48ffa; no voters:
I20260812 21:08:57.049162 25612 raft_consensus.cc:2781] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:57.049345 25612 raft_consensus.cc:686] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 1 LEADER]: Becoming Leader. State: Replica: bdbffa9bc5594328be86882ff6e48ffa, State: Running, Role: LEADER
I20260812 21:08:57.049450 25612 consensus_queue.cc:237] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } }
I20260812 21:08:57.050329 23126 catalog_manager.cc:5495] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa reported cstate change: term changed from 0 to 1, leader changed from <none> to bdbffa9bc5594328be86882ff6e48ffa (127.11.207.130). New cstate: current_term: 1 leader_uuid: "bdbffa9bc5594328be86882ff6e48ffa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } health_report { overall_health: UNKNOWN } } }
W20260812 21:08:57.061677 25019 tablet.cc:2369] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:08:57.069041 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.066736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.11.207.130:35089)
0812 21:08:57.066794 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:57.066796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:57.066806 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:57.066851 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:57.066851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:57.066870 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:57.066938 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:57.067131 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.067611 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.068316 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.068566 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.068634 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.068784 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:57.068888 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:57.068945 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:57.068961 (+ 16us) server_negotiation.cc:299] Negotiation successful
0812 21:08:57.068979 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:08:57.071045 25458 raft_consensus.cc:1261] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Refusing update from remote peer bdbffa9bc5594328be86882ff6e48ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:57.071044 24930 raft_consensus.cc:1261] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Refusing update from remote peer bdbffa9bc5594328be86882ff6e48ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:57.071427 25625 consensus_queue.cc:1035] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:57.071563 25612 consensus_queue.cc:1035] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:57.075986 25633 mvcc.cc:205] Tried to move back new op lower bound from 7317786366240243712 to 7317786366155169792. Current Snapshot: MvccSnapshot[applied={T|T < 7317786366240243712}]
I20260812 21:08:57.076172 25637 mvcc.cc:205] Tried to move back new op lower bound from 7317786366240243712 to 7317786366155169792. Current Snapshot: MvccSnapshot[applied={T|T < 7317786366240243712}]
I20260812 21:08:57.078183 23126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:57.078269 23126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:08:57.080643 23126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 286919065e18466f8495e6f3dae8c02a: Sending DeleteTablet for 3 replicas of tablet 7696a7b22abd43e9a2c94a989abb92d8
I20260812 21:08:57.081022 25425 tablet_service.cc:1514] Processing DeleteTablet for tablet 7696a7b22abd43e9a2c94a989abb92d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:57 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51632
I20260812 21:08:57.081116 24889 tablet_service.cc:1514] Processing DeleteTablet for tablet 7696a7b22abd43e9a2c94a989abb92d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:57 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:59166
I20260812 21:08:57.081323 25155 tablet_service.cc:1514] Processing DeleteTablet for tablet 7696a7b22abd43e9a2c94a989abb92d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:08:57 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:33076
I20260812 21:08:57.081980 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 23062
I20260812 21:08:57.081985 25646 tablet_replica.cc:330] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: stopping tablet replica
I20260812 21:08:57.090603 25646 raft_consensus.cc:2229] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:57.090905 25646 raft_consensus.cc:2258] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:57.094983 25646 ts_tablet_manager.cc:1905] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:57.097214 25646 ts_tablet_manager.cc:1918] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:57.097385 25646 log.cc:1196] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/7696a7b22abd43e9a2c94a989abb92d8
I20260812 21:08:57.097836 25646 ts_tablet_manager.cc:1939] T 7696a7b22abd43e9a2c94a989abb92d8 P 6fb3fbaa178a46ae846877d15304b001: Deleting consensus metadata
I20260812 21:08:57.103348 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.081386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.11.207.188:36971)
0812 21:08:57.081429 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:57.081431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:57.081596 (+ 165us) server_negotiation.cc:407] Connection header received
0812 21:08:57.081689 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:57.081690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:57.081702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:57.081728 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:57.096083 (+ 14355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.097155 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.098181 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.098480 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.098527 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.100212 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:57.100214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:57.100214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:57.100232 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:57.102061 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:57.102492 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:57.102493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:57.102494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:57.102565 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:57.102767 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:57.102767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:57.102768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:57.102922 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:57.103025 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:57.103150 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 21:08:57.103249 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:08:57.104357 25332 connection.cc:535] server connection from 127.0.0.1:51632 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:08:57.104457 25651 tablet_replica.cc:330] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: stopping tablet replica
I20260812 21:08:57.104722 25651 raft_consensus.cc:2229] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:08:57.104866 25651 raft_consensus.cc:2258] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:57.105055 25649 tablet_replica.cc:330] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: stopping tablet replica
I20260812 21:08:57.105178 25649 raft_consensus.cc:2229] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:08:57.105304 25649 raft_consensus.cc:2258] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:08:57.106273 25651 ts_tablet_manager.cc:1905] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:57.108069 25651 ts_tablet_manager.cc:1918] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:57.108152 25651 log.cc:1196] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/7696a7b22abd43e9a2c94a989abb92d8
I20260812 21:08:57.108445 25651 ts_tablet_manager.cc:1939] T 7696a7b22abd43e9a2c94a989abb92d8 P bdbffa9bc5594328be86882ff6e48ffa: Deleting consensus metadata
W20260812 21:08:57.108856 25065 connection.cc:441] server connection from 127.0.0.1:33076 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33076 (request call id 1) could send its response
I20260812 21:08:57.109112 25649 ts_tablet_manager.cc:1905] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:08:57.110652 25649 ts_tablet_manager.cc:1918] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:08:57.110737 25649 log.cc:1196] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/7696a7b22abd43e9a2c94a989abb92d8
I20260812 21:08:57.111012 25649 ts_tablet_manager.cc:1939] T 7696a7b22abd43e9a2c94a989abb92d8 P b3078d23e0d34fe5b5011d6c50e1da9c: Deleting consensus metadata
W20260812 21:08:57.111366 24808 connection.cc:441] server connection from 127.0.0.1:59166 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59166 (request call id 1) could send its response
I20260812 21:08:57.113039 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.105977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.11.207.189:37091)
0812 21:08:57.106026 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:57.106028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:57.106335 (+ 307us) server_negotiation.cc:407] Connection header received
0812 21:08:57.106436 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:57.106437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:57.106454 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:57.106483 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:57.107076 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.107599 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.108390 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.108665 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.108728 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.110367 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:57.110368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:57.110369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:57.110378 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:57.111838 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:57.112147 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:57.112155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:57.112156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:57.112270 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:57.112486 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:57.112487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:57.112488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:57.112648 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:57.112743 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:57.112860 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 21:08:57.112947 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:57.119237 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:57.106289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.11.207.188:36971)
0812 21:08:57.106334 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:57.106335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:57.113097 (+ 6762us) server_negotiation.cc:407] Connection header received
0812 21:08:57.113359 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:57.113361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:57.113374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:57.113446 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:57.113931 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.114697 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.115347 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:57.115660 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:57.115728 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:57.116818 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:57.116820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:57.116820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:57.116829 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:57.118214 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:57.118510 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:57.118510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:57.118511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:57.118572 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:57.118754 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:57.118755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:57.118755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:57.118895 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:57.118985 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:57.119062 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 21:08:57.119139 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:08:58.077104 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.076722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:50609)
0812 21:08:58.076944 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.077039 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
I20260812 21:08:58.077313 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.077001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:45651)
0812 21:08:58.077179 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.077263 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260812 21:08:58.077495 25668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.077250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:40903)
0812 21:08:58.077379 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.077454 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
W20260812 21:08:58.077512 25286 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:34819 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
W20260812 21:08:58.077653 25555 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:34819 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
W20260812 21:08:58.077663 25014 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:34819 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:08:58.580183 25670 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 286919065e18466f8495e6f3dae8c02a)
I20260812 21:08:58.580329 25670 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:08:58.580456 25671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 286919065e18466f8495e6f3dae8c02a)
I20260812 21:08:58.580547 25671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:08:58.580750 25671 leader_election.cc:288] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), f48f780f144f4656959ad5e5fc9f084b (127.11.207.189:37091)
I20260812 21:08:58.580935 25670 leader_election.cc:288] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188:36971)
I20260812 21:08:58.581061 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.580834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33778)
0812 21:08:58.580950 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.581017 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260812 21:08:58.581200 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.580790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33768)
0812 21:08:58.581116 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.581164 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":69,"threads_started":1}
W20260812 21:08:58.581319 24297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
W20260812 21:08:58.581327 24647 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:08:58.581746 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.581662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33792)
0812 21:08:58.581698 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.581721 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:08:58.581858 24297 leader_election.cc:334] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:08:58.581979 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.581763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33802)
0812 21:08:58.581865 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.581946 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 21:08:58.582057 24647 leader_election.cc:334] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:08:58.584545 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.581002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:37091 (local address 127.0.0.1:42458)
0812 21:08:58.581116 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.581126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:58.581184 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:58.581390 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:58.581391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:58.581395 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:58.581455 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:58.581458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.582330 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.582331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:58.583537 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:58.583541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.584275 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.584276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:58.584378 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:58.584383 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:58.584431 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:08:58.584469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":52,"threads_started":1}
I20260812 21:08:58.584547 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.581073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.11.207.189:37091)
0812 21:08:58.581192 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:58.581194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:58.581208 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:08:58.581240 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:58.581241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:58.581253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:58.581329 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:58.581537 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.582254 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.583632 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.584194 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.584267 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:58.584275 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:58.584436 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 21:08:58.584478 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":101,"thread_start_us":37,"threads_started":1}
I20260812 21:08:58.584549 25687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.581185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.11.207.188:36971)
0812 21:08:58.581415 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:58.581417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:58.581424 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:58.581447 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:58.581448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:58.581457 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:58.581672 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:08:58.581708 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.582336 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.583639 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.584193 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.584267 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:58.584275 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:58.584440 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 21:08:58.584483 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":38,"threads_started":1}
I20260812 21:08:58.584553 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.580843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:36971 (local address 127.0.0.1:51610)
0812 21:08:58.581261 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.581270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:58.581325 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:58.581540 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:58.581541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:58.581543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:08:58.581608 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:58.581610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.582403 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.582404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:58.583562 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:58.583565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:58.584270 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:58.584271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:58.584368 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:58.584373 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:08:58.584434 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:08:58.584479 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":4,"thread_start_us":52,"threads_started":1}
I20260812 21:08:58.584975 24338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6354a889622345c2aaafcc5ba75e0ddf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f48f780f144f4656959ad5e5fc9f084b" is_pre_election: true
I20260812 21:08:58.585000 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f48f780f144f4656959ad5e5fc9f084b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6354a889622345c2aaafcc5ba75e0ddf" is_pre_election: true
I20260812 21:08:58.585078 24338 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6354a889622345c2aaafcc5ba75e0ddf in term 1.
I20260812 21:08:58.585085 24693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f48f780f144f4656959ad5e5fc9f084b in term 1.
I20260812 21:08:58.585219 24294 leader_election.cc:302] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6354a889622345c2aaafcc5ba75e0ddf, f48f780f144f4656959ad5e5fc9f084b; no voters: 286919065e18466f8495e6f3dae8c02a
I20260812 21:08:58.585279 24644 leader_election.cc:302] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6354a889622345c2aaafcc5ba75e0ddf, f48f780f144f4656959ad5e5fc9f084b; no voters: 286919065e18466f8495e6f3dae8c02a
I20260812 21:08:58.585319 25670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 21:08:58.585361 25670 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 286919065e18466f8495e6f3dae8c02a)
I20260812 21:08:58.585387 25670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:08:58.585388 25671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 21:08:58.585413 25671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Starting leader election (detected failure of leader 286919065e18466f8495e6f3dae8c02a)
I20260812 21:08:58.585438 25671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:08:58.586266 25670 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:08:58.586272 25671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:08:58.586405 25671 leader_election.cc:288] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 2 election: Requested vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), f48f780f144f4656959ad5e5fc9f084b (127.11.207.189:37091)
I20260812 21:08:58.586405 25670 leader_election.cc:288] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 2 election: Requested vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188:36971)
I20260812 21:08:58.586561 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.586476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33808)
0812 21:08:58.586510 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.586538 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 21:08:58.586575 24338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6354a889622345c2aaafcc5ba75e0ddf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f48f780f144f4656959ad5e5fc9f084b"
I20260812 21:08:58.586673 24338 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6354a889622345c2aaafcc5ba75e0ddf in current term 2: Already voted for candidate f48f780f144f4656959ad5e5fc9f084b in this term.
I20260812 21:08:58.586737 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f48f780f144f4656959ad5e5fc9f084b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6354a889622345c2aaafcc5ba75e0ddf"
I20260812 21:08:58.586831 24693 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f48f780f144f4656959ad5e5fc9f084b in current term 2: Already voted for candidate 6354a889622345c2aaafcc5ba75e0ddf in this term.
I20260812 21:08:58.586833 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.586655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33814)
0812 21:08:58.586779 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.586803 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 21:08:58.587092 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.587001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33818)
0812 21:08:58.587036 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.587067 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 21:08:58.587118 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:58.587041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:33822)
0812 21:08:58.587072 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:58.587096 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 21:08:58.587164 24297 leader_election.cc:334] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
W20260812 21:08:58.587179 24647 leader_election.cc:334] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:08:58.587209 24297 leader_election.cc:302] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f48f780f144f4656959ad5e5fc9f084b; no voters: 286919065e18466f8495e6f3dae8c02a, 6354a889622345c2aaafcc5ba75e0ddf
I20260812 21:08:58.587213 24647 leader_election.cc:302] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6354a889622345c2aaafcc5ba75e0ddf; no voters: 286919065e18466f8495e6f3dae8c02a, f48f780f144f4656959ad5e5fc9f084b
I20260812 21:08:58.587297 25671 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 21:08:58.587302 25670 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 21:08:59.078117 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:59.077847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:55959)
0812 21:08:59.078024 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:59.078066 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260812 21:08:59.078118 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:59.077912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:51333)
0812 21:08:59.078023 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:59.078064 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":48,"threads_started":1}
I20260812 21:08:59.078132 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:59.077925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:43539)
0812 21:08:59.078054 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:59.078091 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260812 21:09:00.078773 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.078532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:35945)
0812 21:09:00.078680 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.078720 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260812 21:09:00.078855 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.078646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:60717)
0812 21:09:00.078762 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.078806 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
I20260812 21:09:00.078948 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.078765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:46137)
0812 21:09:00.078877 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.078907 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260812 21:09:00.440237 25705 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:00.440374 25705 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:09:00.440531 25705 leader_election.cc:288] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188:36971)
I20260812 21:09:00.440791 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f48f780f144f4656959ad5e5fc9f084b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6354a889622345c2aaafcc5ba75e0ddf" is_pre_election: true
I20260812 21:09:00.440863 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.440638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:34102)
0812 21:09:00.440765 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.440802 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260812 21:09:00.440899 24693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f48f780f144f4656959ad5e5fc9f084b in term 2.
I20260812 21:09:00.441062 24294 leader_election.cc:302] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6354a889622345c2aaafcc5ba75e0ddf, f48f780f144f4656959ad5e5fc9f084b; no voters:
I20260812 21:09:00.441170 25705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 21:09:00.441222 25705 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:00.441246 25705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:09:00.441344 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.441231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:34104)
0812 21:09:00.441282 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.441318 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 21:09:00.441422 24297 leader_election.cc:334] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:09:00.442044 25705 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:09:00.442162 25705 leader_election.cc:288] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 3 election: Requested vote from peers 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819), 6354a889622345c2aaafcc5ba75e0ddf (127.11.207.188:36971)
I20260812 21:09:00.442348 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.442252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:34116)
0812 21:09:00.442291 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.442323 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 21:09:00.442351 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f48f780f144f4656959ad5e5fc9f084b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6354a889622345c2aaafcc5ba75e0ddf"
I20260812 21:09:00.442427 24693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:09:00.442672 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.442568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:34128)
0812 21:09:00.442618 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.442649 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 21:09:00.442809 24297 leader_election.cc:334] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
I20260812 21:09:00.443245 24693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f48f780f144f4656959ad5e5fc9f084b in term 3.
I20260812 21:09:00.443403 24294 leader_election.cc:302] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6354a889622345c2aaafcc5ba75e0ddf, f48f780f144f4656959ad5e5fc9f084b; no voters: 286919065e18466f8495e6f3dae8c02a
I20260812 21:09:00.443495 25705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 3 FOLLOWER]: Leader election won for term 3
I20260812 21:09:00.443614 25705 raft_consensus.cc:686] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [term 3 LEADER]: Becoming Leader. State: Replica: f48f780f144f4656959ad5e5fc9f084b, State: Running, Role: LEADER
I20260812 21:09:00.443686 25705 consensus_queue.cc:237] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } }
I20260812 21:09:00.444034 25713 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f48f780f144f4656959ad5e5fc9f084b. Latest consensus state: current_term: 3 leader_uuid: "f48f780f144f4656959ad5e5fc9f084b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:09:00.444125 25713 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:00.444324 25716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:00.444820 25716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:00.444904 25716 catalog_manager.cc:1259] Loaded cluster ID: c502e73b744e4a8da3609363b00bb5eb
I20260812 21:09:00.444936 25716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:09:00.445112 25716 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:00.445183 25716 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Loaded TSK: 0
I20260812 21:09:00.445325 25716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:09:00.519718 24321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:09:00.521355 24321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:00.522298 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.522177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:34138)
0812 21:09:00.522226 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.522265 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:09:00.522341 24693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [term 3 FOLLOWER]: Refusing update from remote peer f48f780f144f4656959ad5e5fc9f084b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 21:09:00.522548 25713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 21:09:00.522637 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.522545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.0.0.1:34154)
0812 21:09:00.522592 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.522618 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 21:09:00.522706 24297 consensus_peers.cc:487] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b -> Peer 286919065e18466f8495e6f3dae8c02a (127.11.207.190:34819): Couldn't send request to peer 286919065e18466f8495e6f3dae8c02a. Status: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 21:09:00.523135 25720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f48f780f144f4656959ad5e5fc9f084b. Latest consensus state: current_term: 3 leader_uuid: "f48f780f144f4656959ad5e5fc9f084b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:09:00.523245 25720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:09:00.523718 25713 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f48f780f144f4656959ad5e5fc9f084b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:09:00.523810 25713 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:00.523835 25721 mvcc.cc:205] Tried to move back new op lower bound from 7317786380377518080 to 7317786380058030080. Current Snapshot: MvccSnapshot[applied={T|T < 7317786380377518080}]
I20260812 21:09:00.524326 25720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f48f780f144f4656959ad5e5fc9f084b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "286919065e18466f8495e6f3dae8c02a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34819 } } peers { permanent_uuid: "f48f780f144f4656959ad5e5fc9f084b" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 37091 } attrs { promote: false } } peers { permanent_uuid: "6354a889622345c2aaafcc5ba75e0ddf" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 36971 } attrs { promote: false } } }
I20260812 21:09:00.524518 25720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6354a889622345c2aaafcc5ba75e0ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:09:00.528664 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.525357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:42515 (local address 127.0.0.1:51642)
0812 21:09:00.525542 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.525552 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:00.525606 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:00.525784 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:00.525786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:00.525788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:00.525847 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:00.525848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.526837 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.526839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:00.527687 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:00.527690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.528421 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:00.528487 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.528491 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:00.528545 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:09:00.528586 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":36}
I20260812 21:09:00.528673 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.525420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.11.207.131:42515)
0812 21:09:00.525641 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:00.525643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:00.525652 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:00.525689 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:00.525690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:00.525701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:00.525768 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:00.525926 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.526739 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.527772 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.528335 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528427 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.528435 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:00.528555 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 21:09:00.528592 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":61,"threads_started":1}
I20260812 21:09:00.528942 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.525359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40121 (local address 127.0.0.1:59182)
0812 21:09:00.525599 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.525610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:00.525661 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:00.525850 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:00.525851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:00.525852 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:00.525897 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:00.525898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.526797 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.526798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:00.528034 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:00.528037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.528756 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:00.528833 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.528838 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:09:00.528865 (+ 27us) client_negotiation.cc:240] Negotiation successful
0812 21:09:00.528898 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":85,"thread_start_us":66,"threads_started":1}
I20260812 21:09:00.529109 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.525445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:35089 (local address 127.0.0.1:33094)
0812 21:09:00.525788 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.525797 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:00.525845 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:00.526050 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:00.526051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:00.526053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:00.526087 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:00.526088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.526770 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.526771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:00.528044 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:00.528046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.528898 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:00.528956 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.528960 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:00.529001 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 21:09:00.529036 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":52,"thread_start_us":56,"threads_started":1}
I20260812 21:09:00.529166 25425 tablet_service.cc:1467] Processing CreateTablet for tablet 157556b7199c466b90cb226a8718c70d (DEFAULT_TABLE table=test-table [id=2200e328f2fc43a8b48c265f7941c463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:00.529247 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.525621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59182 (local address 127.11.207.129:40121)
0812 21:09:00.525724 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:00.525725 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:00.525733 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:09:00.525759 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:00.525760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:00.525770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:00.525849 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:00.525949 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.526691 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528110 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.528656 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.529108 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.529118 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:00.529137 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 21:09:00.529181 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
I20260812 21:09:00.529300 25425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 157556b7199c466b90cb226a8718c70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:00.529354 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.525525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.11.207.130:35089)
0812 21:09:00.525746 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:00.525748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:00.525958 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:09:00.525981 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:00.525981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:00.525991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:00.526039 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:00.526140 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.526699 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528235 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.528796 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.528860 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.528867 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:00.529246 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 21:09:00.529289 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":44,"threads_started":1}
I20260812 21:09:00.529513 24889 tablet_service.cc:1467] Processing CreateTablet for tablet 157556b7199c466b90cb226a8718c70d (DEFAULT_TABLE table=test-table [id=2200e328f2fc43a8b48c265f7941c463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:00.529632 24889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 157556b7199c466b90cb226a8718c70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:00.530799 25739 tablet_bootstrap.cc:492] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Bootstrap starting.
I20260812 21:09:00.531015 25740 tablet_bootstrap.cc:492] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: Bootstrap starting.
I20260812 21:09:00.531214 25739 tablet_bootstrap.cc:654] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:00.531234 25155 tablet_service.cc:1467] Processing CreateTablet for tablet 157556b7199c466b90cb226a8718c70d (DEFAULT_TABLE table=test-table [id=2200e328f2fc43a8b48c265f7941c463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:00.531344 25155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 157556b7199c466b90cb226a8718c70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:00.531414 25740 tablet_bootstrap.cc:654] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:00.532007 25740 tablet_bootstrap.cc:492] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: No bootstrap required, opened a new log
I20260812 21:09:00.532065 25740 ts_tablet_manager.cc:1397] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:00.532220 25740 raft_consensus.cc:348] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.532297 25740 raft_consensus.cc:374] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:00.532315 25740 raft_consensus.cc:729] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3078d23e0d34fe5b5011d6c50e1da9c, State: Initialized, Role: FOLLOWER
I20260812 21:09:00.532377 25740 consensus_queue.cc:260] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.532589 25740 ts_tablet_manager.cc:1428] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:09:00.532725 25739 tablet_bootstrap.cc:492] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: No bootstrap required, opened a new log
I20260812 21:09:00.532775 25739 ts_tablet_manager.cc:1397] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:00.532894 25748 tablet_bootstrap.cc:492] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: Bootstrap starting.
I20260812 21:09:00.532943 25739 raft_consensus.cc:348] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.533002 25739 raft_consensus.cc:374] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:00.533021 25739 raft_consensus.cc:729] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb3fbaa178a46ae846877d15304b001, State: Initialized, Role: FOLLOWER
I20260812 21:09:00.533008 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.532891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:52547)
0812 21:09:00.532941 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.532982 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 21:09:00.533063 25739 consensus_queue.cc:260] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.533247 25739 ts_tablet_manager.cc:1428] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:00.533259 25748 tablet_bootstrap.cc:654] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:00.533331 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.533256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:41593)
0812 21:09:00.533289 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.533310 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 21:09:00.533799 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.533702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:38783)
0812 21:09:00.533740 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.533769 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 21:09:00.534096 25561 heartbeater.cc:503] Master 127.11.207.189:37091 was elected leader, sending a full tablet report...
I20260812 21:09:00.534241 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.534161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:48703)
0812 21:09:00.534191 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.534215 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 21:09:00.534360 25748 tablet_bootstrap.cc:492] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: No bootstrap required, opened a new log
I20260812 21:09:00.534417 25748 ts_tablet_manager.cc:1397] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:00.534570 25748 raft_consensus.cc:348] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.534637 25748 raft_consensus.cc:374] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:00.534651 25748 raft_consensus.cc:729] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbffa9bc5594328be86882ff6e48ffa, State: Initialized, Role: FOLLOWER
I20260812 21:09:00.534626 25018 heartbeater.cc:503] Master 127.11.207.189:37091 was elected leader, sending a full tablet report...
I20260812 21:09:00.534693 25748 consensus_queue.cc:260] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.534894 25748 ts_tablet_manager.cc:1428] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:00.535151 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.535057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:35551)
0812 21:09:00.535101 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.535124 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 21:09:00.535503 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.535393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:43465)
0812 21:09:00.535450 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.535480 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:09:00.535523 25289 heartbeater.cc:503] Master 127.11.207.189:37091 was elected leader, sending a full tablet report...
I20260812 21:09:00.580928 25752 raft_consensus.cc:482] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:00.581033 25752 raft_consensus.cc:504] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.581218 25752 leader_election.cc:288] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121), 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515)
I20260812 21:09:00.581511 25458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "157556b7199c466b90cb226a8718c70d" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb3fbaa178a46ae846877d15304b001" is_pre_election: true
I20260812 21:09:00.581511 24930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "157556b7199c466b90cb226a8718c70d" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" is_pre_election: true
I20260812 21:09:00.581624 25458 raft_consensus.cc:2454] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 0.
I20260812 21:09:00.581624 24930 raft_consensus.cc:2454] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 0.
I20260812 21:09:00.581784 25063 leader_election.cc:302] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3078d23e0d34fe5b5011d6c50e1da9c, bdbffa9bc5594328be86882ff6e48ffa; no voters:
I20260812 21:09:00.581907 25752 raft_consensus.cc:2781] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:00.581947 25752 raft_consensus.cc:482] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:00.581960 25752 raft_consensus.cc:3037] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:00.582700 25752 raft_consensus.cc:504] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.582828 25752 leader_election.cc:288] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 election: Requested vote from peers b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121), 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515)
I20260812 21:09:00.583027 24930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "157556b7199c466b90cb226a8718c70d" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c"
I20260812 21:09:00.583037 25458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "157556b7199c466b90cb226a8718c70d" candidate_uuid: "bdbffa9bc5594328be86882ff6e48ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb3fbaa178a46ae846877d15304b001"
I20260812 21:09:00.583096 24930 raft_consensus.cc:3037] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:00.583125 25458 raft_consensus.cc:3037] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:00.583905 24930 raft_consensus.cc:2454] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 1.
I20260812 21:09:00.583987 25458 raft_consensus.cc:2454] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdbffa9bc5594328be86882ff6e48ffa in term 1.
I20260812 21:09:00.584090 25063 leader_election.cc:302] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3078d23e0d34fe5b5011d6c50e1da9c, bdbffa9bc5594328be86882ff6e48ffa; no voters:
I20260812 21:09:00.584224 25752 raft_consensus.cc:2781] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:00.584414 25752 raft_consensus.cc:686] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 1 LEADER]: Becoming Leader. State: Replica: bdbffa9bc5594328be86882ff6e48ffa, State: Running, Role: LEADER
I20260812 21:09:00.584471 25752 consensus_queue.cc:237] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } }
I20260812 21:09:00.584740 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.584621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:56275)
0812 21:09:00.584664 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.584707 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 21:09:00.585136 24322 catalog_manager.cc:5495] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa reported cstate change: term changed from 0 to 1, leader changed from <none> to bdbffa9bc5594328be86882ff6e48ffa (127.11.207.130). New cstate: current_term: 1 leader_uuid: "bdbffa9bc5594328be86882ff6e48ffa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdbffa9bc5594328be86882ff6e48ffa" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 35089 } health_report { overall_health: HEALTHY } } }
I20260812 21:09:00.602914 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.600628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.11.207.130:35089)
0812 21:09:00.600690 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:00.600692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:00.600717 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:09:00.600787 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:00.600788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:00.600798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:00.600839 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:00.600968 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.601454 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.602252 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:00.602460 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:00.602513 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:00.602710 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:00.602738 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:00.602769 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:00.602817 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 21:09:00.602837 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:09:00.603936 24930 raft_consensus.cc:1261] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Refusing update from remote peer bdbffa9bc5594328be86882ff6e48ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:00.603935 25458 raft_consensus.cc:1261] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Refusing update from remote peer bdbffa9bc5594328be86882ff6e48ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:00.604173 25753 consensus_queue.cc:1035] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3078d23e0d34fe5b5011d6c50e1da9c" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:00.604307 25752 consensus_queue.cc:1035] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb3fbaa178a46ae846877d15304b001" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:00.604912 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.604767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:53731)
0812 21:09:00.604827 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.604862 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 21:09:00.605475 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.605362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:56717)
0812 21:09:00.605403 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.605441 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 21:09:00.605686 25760 mvcc.cc:205] Tried to move back new op lower bound from 7317786380711903232 to 7317786380634468352. Current Snapshot: MvccSnapshot[applied={T|T < 7317786380711903232}]
I20260812 21:09:00.606142 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.605988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:42537)
0812 21:09:00.606086 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.606112 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 21:09:00.606192 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.606095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.129:51041)
0812 21:09:00.606131 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.606164 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 21:09:00.607280 24321 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:00.607384 24321 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:00.607446 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.607359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.130:47113)
0812 21:09:00.607400 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.607423 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:09:00.608290 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.608194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34819 (local address 127.11.207.131:44889)
0812 21:09:00.608235 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.608260 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:09:00.608621 24321 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f48f780f144f4656959ad5e5fc9f084b: Sending DeleteTablet for 3 replicas of tablet 157556b7199c466b90cb226a8718c70d
I20260812 21:09:00.609053 24889 tablet_service.cc:1514] Processing DeleteTablet for tablet 157556b7199c466b90cb226a8718c70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:00 UTC) from {username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:59182
I20260812 21:09:00.609198 25425 tablet_service.cc:1514] Processing DeleteTablet for tablet 157556b7199c466b90cb226a8718c70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:00 UTC) from {username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:51642
I20260812 21:09:00.609241 25155 tablet_service.cc:1514] Processing DeleteTablet for tablet 157556b7199c466b90cb226a8718c70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:00 UTC) from {username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:33094
I20260812 21:09:00.609256 25774 tablet_replica.cc:330] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: stopping tablet replica
I20260812 21:09:00.609325 25774 raft_consensus.cc:2229] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:00.609365 25774 raft_consensus.cc:2258] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:00.609365 25421 tablet_service.cc:1514] Processing DeleteTablet for tablet 157556b7199c466b90cb226a8718c70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:00 UTC) from {username='oryx', principal='oryx/127.11.207.189@KRBTEST.COM'} at 127.0.0.1:51642
I20260812 21:09:00.609390 25775 tablet_replica.cc:330] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: stopping tablet replica
I20260812 21:09:00.609579 25775 raft_consensus.cc:2229] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:00.609627 25774 ts_tablet_manager.cc:1905] T 157556b7199c466b90cb226a8718c70d P b3078d23e0d34fe5b5011d6c50e1da9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:00.609637 25777 tablet_replica.cc:330] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: stopping tablet replica
I20260812 21:09:00.609696 25777 raft_consensus.cc:2229] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:09:00.609766 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 24595
I20260812 21:09:00.609671 25775 raft_consensus.cc:2258] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:00.610667 25775 ts_tablet_manager.cc:1905] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:00.613272 25775 ts_tablet_manager.cc:1918] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:00.613327 25775 log.cc:1196] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/157556b7199c466b90cb226a8718c70d
I20260812 21:09:00.613528 25775 ts_tablet_manager.cc:1939] T 157556b7199c466b90cb226a8718c70d P 6fb3fbaa178a46ae846877d15304b001: Deleting consensus metadata
I20260812 21:09:00.613883 24294 catalog_manager.cc:4841] TS 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515): tablet 157556b7199c466b90cb226a8718c70d (table test-table [id=2200e328f2fc43a8b48c265f7941c463]) successfully deleted
W20260812 21:09:00.614073 24294 catalog_manager.cc:4808] TS 6fb3fbaa178a46ae846877d15304b001 (127.11.207.131:42515): delete failed for tablet 157556b7199c466b90cb226a8718c70d because the tablet was not found. No further retry: Not found: Tablet not found: 157556b7199c466b90cb226a8718c70d
I20260812 21:09:00.609814 25777 raft_consensus.cc:2258] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:00.621086 25777 ts_tablet_manager.cc:1905] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:00.622536 25777 ts_tablet_manager.cc:1918] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:00.622602 25777 log.cc:1196] T 157556b7199c466b90cb226a8718c70d P bdbffa9bc5594328be86882ff6e48ffa: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/157556b7199c466b90cb226a8718c70d
I20260812 21:09:00.622964 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 25027
I20260812 21:09:00.623126 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.622993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40121 (local address 127.0.0.1:59184)
0812 21:09:00.623058 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.623091 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 21:09:00.631191 24293 catalog_manager.cc:4584] TS b3078d23e0d34fe5b5011d6c50e1da9c (127.11.207.129:40121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 157556b7199c466b90cb226a8718c70d: Network error: Client connection negotiation failed: client connection to 127.11.207.129:40121: connect: Connection refused (error 111)
I20260812 21:09:00.636422 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:00.636272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:35089 (local address 127.0.0.1:33112)
0812 21:09:00.636331 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:00.636381 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.130:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:09:00.636428 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 25299
I20260812 21:09:00.658633 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 24030
I20260812 21:09:00.667771 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 24417
2026-08-12T21:09:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6798 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:09:00 dist-test-slave-lkx2 krb5kdc[25784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:09:00 dist-test-slave-lkx2 krb5kdc[25784](info): set up 2 sockets
Aug 12 21:09:00 dist-test-slave-lkx2 krb5kdc[25784](info): commencing operation
krb5kdc: starting...
W20260812 21:09:01.872598 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:09:01 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab.
Entry for principal oryx/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab.
I20260812 21:09:01.995121 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43237
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab
--principal=oryx/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:02.078995 25807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:02.079232 25807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:02.079263 25807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:02.080489 25807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:02.080533 25807 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 21:09:02.080550 25807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:02.080566 25807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:02.080583 25807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:02.080605 25807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:02.082257 25807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43237
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.190.keytab
--principal=oryx/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:02.082521 25807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:02.082794 25807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:02.088166 25816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:02.088198 25823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:02.088310 25815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:02.088187 25807 server_base.cc:1034] running on GCE node
I20260812 21:09:02.088698 25807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:02.088990 25807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:02.090147 25807 hybrid_clock.cc:648] HybridClock initialized: now 1786568942090122 us; error 43 us; skew 500 ppm
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:02.092736 25807 init.cc:377] Logged in from keytab as oryx/127.11.207.190@KRBTEST.COM (short username oryx)
I20260812 21:09:02.093200 25807 webserver.cc:460] Webserver started at http://127.11.207.190:39503/ using document root <none> and password file <none>
I20260812 21:09:02.093441 25807 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:02.093499 25807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:02.093644 25807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:02.094801 25807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "077c37c9c2d44c5ab10d323c350d1133"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "108285918d2e36aacac0a37a40fb51b1"
server_key_iv: "57233258688fbdfdd0170332beeeb8fe"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.095161 25807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "077c37c9c2d44c5ab10d323c350d1133"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "108285918d2e36aacac0a37a40fb51b1"
server_key_iv: "57233258688fbdfdd0170332beeeb8fe"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.096532 25807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:09:02.097255 25839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.097434 25807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:02.097527 25807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "077c37c9c2d44c5ab10d323c350d1133"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "108285918d2e36aacac0a37a40fb51b1"
server_key_iv: "57233258688fbdfdd0170332beeeb8fe"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.097584 25807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:02.108187 25807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:02.110272 25807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:02.110390 25807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:02.114953 25807 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43237
I20260812 21:09:02.115054 25947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43237 every 8 connection(s)
I20260812 21:09:02.115341 25807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:09:02.116081 25948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:02.118537 25948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Bootstrap starting.
I20260812 21:09:02.119210 25948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:02.119553 25948 log.cc:824] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:02.120123 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 25807
I20260812 21:09:02.120252 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:09:02.120393 25948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: No bootstrap required, opened a new log
I20260812 21:09:02.120491 12094 external_mini_cluster.cc:1428] Setting key 3aa8afbba7041c80e0ea89506ad17b9b
I20260812 21:09:02.122743 25948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077c37c9c2d44c5ab10d323c350d1133" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43237 } }
I20260812 21:09:02.122892 25948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:02.122926 25948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 077c37c9c2d44c5ab10d323c350d1133, State: Initialized, Role: FOLLOWER
I20260812 21:09:02.123052 25948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077c37c9c2d44c5ab10d323c350d1133" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43237 } }
I20260812 21:09:02.123131 25948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:09:02.123169 25948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:09:02.123216 25948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:02.123785 25948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077c37c9c2d44c5ab10d323c350d1133" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43237 } }
I20260812 21:09:02.123885 25948 leader_election.cc:302] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 077c37c9c2d44c5ab10d323c350d1133; no voters:
I20260812 21:09:02.124050 25948 leader_election.cc:288] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:09:02.124096 25961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:02.124223 25961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [term 1 LEADER]: Becoming Leader. State: Replica: 077c37c9c2d44c5ab10d323c350d1133, State: Running, Role: LEADER
I20260812 21:09:02.124315 25948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:09:02.124308 25961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077c37c9c2d44c5ab10d323c350d1133" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43237 } }
I20260812 21:09:02.124722 25961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 077c37c9c2d44c5ab10d323c350d1133. Latest consensus state: current_term: 1 leader_uuid: "077c37c9c2d44c5ab10d323c350d1133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077c37c9c2d44c5ab10d323c350d1133" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43237 } } }
I20260812 21:09:02.124929 25961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:02.124738 25964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "077c37c9c2d44c5ab10d323c350d1133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077c37c9c2d44c5ab10d323c350d1133" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43237 } } }
I20260812 21:09:02.125154 25964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:02.125197 25974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:02.125713 25974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:02.127429 25974 catalog_manager.cc:1347] Generated new cluster ID: bd46a7c4a3d5468c9e7c94ad8ebefabd
I20260812 21:09:02.127485 25974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:02.132264 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.121139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.11.207.190:43237)
0812 21:09:02.121378 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.121386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.121414 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:09:02.122714 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.122723 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.122784 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.122940 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.123645 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.124915 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.125996 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.126678 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.126796 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.129264 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.129268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.129269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.129282 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.130810 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.131142 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.131145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.131146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.131249 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.131432 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.131433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.131433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.131677 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.131838 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.131935 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.132073 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
I20260812 21:09:02.148180 25974 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:09:02.148798 25974 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:02.154098 25974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Generated new TSK 0
I20260812 21:09:02.154294 25974 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab.
Entry for principal oryx/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab.
I20260812 21:09:02.186846 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43237
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:02.272226 26008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:02.272497 26008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:02.272529 26008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:02.273711 26008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:02.273751 26008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:02.273772 26008 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:09:02.273801 26008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:09:02.275314 26008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:43237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:02.275542 26008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:02.275754 26008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:02.280747 26023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:02.280742 26022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:02.280810 26008 server_base.cc:1034] running on GCE node
W20260812 21:09:02.280769 26028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:02.281158 26008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:02.281394 26008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:02.282569 26008 hybrid_clock.cc:648] HybridClock initialized: now 1786568942282549 us; error 30 us; skew 500 ppm
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:02.285096 26008 init.cc:377] Logged in from keytab as oryx/127.11.207.129@KRBTEST.COM (short username oryx)
I20260812 21:09:02.285559 26008 webserver.cc:460] Webserver started at http://127.11.207.129:32805/ using document root <none> and password file <none>
I20260812 21:09:02.285742 26008 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:02.285781 26008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:02.285876 26008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:02.286883 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "8c7311a477444d3f9c86d6a12225e9f2"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "fd7f87d9cef29ad2a2d7ce762362d279"
server_key_iv: "d5b41e858426e5787d7cae105f08f741"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.287240 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "8c7311a477444d3f9c86d6a12225e9f2"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "fd7f87d9cef29ad2a2d7ce762362d279"
server_key_iv: "d5b41e858426e5787d7cae105f08f741"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.288549 26008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:09:02.289271 26039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.289459 26008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:02.289544 26008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "8c7311a477444d3f9c86d6a12225e9f2"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "fd7f87d9cef29ad2a2d7ce762362d279"
server_key_iv: "d5b41e858426e5787d7cae105f08f741"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.289601 26008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:02.313989 26008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:02.315845 26008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:02.315964 26008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:02.316551 26008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:02.316601 26008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.316624 26008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:02.316635 26008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:02.323765 26008 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:33559
I20260812 21:09:02.323896 26246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:33559 every 8 connection(s)
I20260812 21:09:02.324150 26008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:09:02.326128 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.317046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:38631 (local address 127.11.207.190:43237)
0812 21:09:02.317110 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.317112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.317351 (+ 239us) server_negotiation.cc:407] Connection header received
0812 21:09:02.318056 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.318057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.318074 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.318118 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.319032 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.319820 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.320880 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.321101 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.321158 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.323499 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.323500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.323501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.323509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.324948 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.325340 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.325341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.325341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.325421 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.325645 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.325646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.325646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.325740 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.325780 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.326000 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.326064 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:09:02.326279 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.317015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.11.207.129:38631)
0812 21:09:02.317207 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.317260 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.317947 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.318214 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.318219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.318616 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:02.318924 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.318934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.319914 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.319915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.320790 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.320794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.321192 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.321193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.321301 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.321805 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:02.321820 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:02.323408 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:02.325023 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.325027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.325029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.325277 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.325473 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.325474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.325474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.325591 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.325781 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:02.325785 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:02.325897 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.325980 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.326096 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260812 21:09:02.326761 26247 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43237
I20260812 21:09:02.326871 26247 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:02.327059 26247 heartbeater.cc:511] Master 127.11.207.190:43237 requested a full tablet report, sending...
I20260812 21:09:02.327584 25869 ts_manager.cc:194] Registered new tserver with Master: 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559)
I20260812 21:09:02.328262 25869 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:38631
I20260812 21:09:02.332702 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.327827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:40201 (local address 127.11.207.190:43237)
0812 21:09:02.327869 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.327871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.327981 (+ 110us) server_negotiation.cc:407] Connection header received
0812 21:09:02.328033 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.328034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.328043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.328086 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.328386 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.328974 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.329698 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.329880 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.329933 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.330782 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.330783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.330783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.330789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.331794 (+ 1005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.332077 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.332077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.332078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.332118 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.332282 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.332283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.332283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.332417 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.332464 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.332582 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.332642 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:09:02.332718 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.327766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.11.207.129:40201)
0812 21:09:02.327921 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.327931 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.327981 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.328090 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.328090 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.328250 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:02.328299 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.328301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.329093 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.329094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.329627 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.329628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.329965 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.329966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.330036 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.330430 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:02.330435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:02.330709 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:02.331866 (+ 1157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.331867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.331867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.332023 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.332163 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.332164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.332164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.332241 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.332475 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:02.332475 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:02.332514 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.332558 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.332647 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260812 21:09:02.333361 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 26008
I20260812 21:09:02.333483 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:09:02.333698 12094 external_mini_cluster.cc:1428] Setting key d755adf3e4d8b0f888fde45c0948f853
WARNING: no policy specified for oryx/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab.
Entry for principal oryx/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab.
I20260812 21:09:02.358661 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43237
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:02.441488 26264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:02.441713 26264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:02.441747 26264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:02.442922 26264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:02.442968 26264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:02.442996 26264 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:09:02.443018 26264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:09:02.445060 26264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:43237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:02.445320 26264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:02.445541 26264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:02.450484 26272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:02.450531 26275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:02.450528 26273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:02.450685 26264 server_base.cc:1034] running on GCE node
I20260812 21:09:02.450939 26264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:02.451171 26264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:02.452319 26264 hybrid_clock.cc:648] HybridClock initialized: now 1786568942452305 us; error 31 us; skew 500 ppm
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:02.454999 26264 init.cc:377] Logged in from keytab as oryx/127.11.207.130@KRBTEST.COM (short username oryx)
I20260812 21:09:02.455478 26264 webserver.cc:460] Webserver started at http://127.11.207.130:36163/ using document root <none> and password file <none>
I20260812 21:09:02.455693 26264 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:02.455746 26264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:02.455881 26264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:02.457043 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "9bd7a9f9ee08448994f1488caebe319b"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "b66af1f2abcf0d17eb3328cfee723926"
server_key_iv: "8b91ce9e843c1f37cf4f6276d7987f71"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.457417 26264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "9bd7a9f9ee08448994f1488caebe319b"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "b66af1f2abcf0d17eb3328cfee723926"
server_key_iv: "8b91ce9e843c1f37cf4f6276d7987f71"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.458801 26264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:09:02.459514 26286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.459662 26264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:02.459760 26264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "9bd7a9f9ee08448994f1488caebe319b"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "b66af1f2abcf0d17eb3328cfee723926"
server_key_iv: "8b91ce9e843c1f37cf4f6276d7987f71"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.459822 26264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:02.480593 26264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:02.482918 26264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:02.483069 26264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:02.483628 26264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:02.483673 26264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.483708 26264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:02.483732 26264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:02.490576 26264 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:40477
I20260812 21:09:02.490752 26489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:40477 every 8 connection(s)
I20260812 21:09:02.490981 26264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:09:02.493952 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.484226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:40841 (local address 127.11.207.190:43237)
0812 21:09:02.484297 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.484299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.484579 (+ 280us) server_negotiation.cc:407] Connection header received
0812 21:09:02.485187 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.485189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.485205 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.485273 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.486221 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.486973 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.488328 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.488591 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.488660 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.491105 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.491107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.491107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.491117 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.492639 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.493035 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.493036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.493036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.493102 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.493360 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.493360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.493361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.493485 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.493537 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.493818 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.493885 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:09:02.494131 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.484192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.11.207.130:40841)
0812 21:09:02.484437 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.484486 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.485056 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.485382 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.485388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.485795 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:02.486106 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.486113 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.487088 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.487090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.488230 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.488234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.488684 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.488685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.488823 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.489377 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:02.489388 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:02.490994 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:02.492721 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.492728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.492730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.492967 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.493165 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.493166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.493166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.493303 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.493532 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:02.493536 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:02.493705 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.493790 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.493919 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260812 21:09:02.494488 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 26264
I20260812 21:09:02.494592 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:09:02.494613 26491 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43237
I20260812 21:09:02.494789 26491 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:02.494814 12094 external_mini_cluster.cc:1428] Setting key 9c40dbd881e5273dc11902e5c458130c
I20260812 21:09:02.494985 26491 heartbeater.cc:511] Master 127.11.207.190:43237 requested a full tablet report, sending...
I20260812 21:09:02.495406 25869 ts_manager.cc:194] Registered new tserver with Master: 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477)
I20260812 21:09:02.495831 25869 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:40841
I20260812 21:09:02.501307 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.495688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.11.207.130:52109)
0812 21:09:02.495848 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.495860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.495935 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.496065 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.496065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.496215 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:02.496268 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.496269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.497063 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.497064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.497634 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.497636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.498031 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.498033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.498102 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.498549 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:02.498552 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:02.498818 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:02.500232 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.500233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.500234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.500448 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.500682 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.500682 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.500682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.500731 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.501005 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:02.501006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:02.501058 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.501134 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.501228 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":34,"thread_start_us":62,"threads_started":1}
I20260812 21:09:02.501338 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.495742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:52109 (local address 127.11.207.190:43237)
0812 21:09:02.495785 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.495787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.495910 (+ 123us) server_negotiation.cc:407] Connection header received
0812 21:09:02.495987 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.495989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.496000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.496059 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.496330 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.496966 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.497726 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.497963 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.498019 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.498896 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.498898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.498898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.498905 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.500160 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.500549 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.500550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.500550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.500609 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.500786 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.500787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.500787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.500941 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.501007 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.501168 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.501245 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for oryx/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab.
Entry for principal oryx/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab.
I20260812 21:09:02.521406 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43237
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:02.605082 26504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:02.605294 26504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:02.605326 26504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:02.606469 26504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:02.606503 26504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:02.606521 26504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:09:02.606536 26504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:09:02.608073 26504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.207.190:43237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:02.608275 26504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:02.608549 26504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:02.613538 26522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:02.613575 26517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:02.613554 26504 server_base.cc:1034] running on GCE node
W20260812 21:09:02.613538 26516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:02.613972 26504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:02.614192 26504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:02.615338 26504 hybrid_clock.cc:648] HybridClock initialized: now 1786568942615318 us; error 32 us; skew 500 ppm
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:02.617952 26504 init.cc:377] Logged in from keytab as oryx/127.11.207.131@KRBTEST.COM (short username oryx)
I20260812 21:09:02.618417 26504 webserver.cc:460] Webserver started at http://127.11.207.131:46223/ using document root <none> and password file <none>
I20260812 21:09:02.618666 26504 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:02.618722 26504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:02.618862 26504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:02.620038 26504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "50030708162549f587620cc6cb921118"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "9c352da5b8fdd3104c2e1e3cbcf43d5f"
server_key_iv: "a7c1ac578951fef0f85b372aa8d1359b"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.620468 26504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "50030708162549f587620cc6cb921118"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "9c352da5b8fdd3104c2e1e3cbcf43d5f"
server_key_iv: "a7c1ac578951fef0f85b372aa8d1359b"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.621889 26504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:02.622596 26534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.622776 26504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:02.622856 26504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "50030708162549f587620cc6cb921118"
format_stamp: "Formatted at 2026-08-12 21:09:02 on dist-test-slave-lkx2"
server_key: "9c352da5b8fdd3104c2e1e3cbcf43d5f"
server_key_iv: "a7c1ac578951fef0f85b372aa8d1359b"
server_key_version: "encryptionkey@0"
I20260812 21:09:02.622922 26504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:02.644946 26504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:02.646938 26504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:02.647107 26504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:02.647706 26504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:02.647748 26504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.647779 26504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:02.647795 26504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:02.654798 26504 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:44701
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:02.654892 26743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:44701 every 8 connection(s)
I20260812 21:09:02.655187 26504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:09:02.657460 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 26504
I20260812 21:09:02.657569 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:09:02.657753 12094 external_mini_cluster.cc:1428] Setting key b61f078f92d7f93a6604341696de1775
I20260812 21:09:02.657999 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.648175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:43439 (local address 127.11.207.190:43237)
0812 21:09:02.648265 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.648267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.648455 (+ 188us) server_negotiation.cc:407] Connection header received
0812 21:09:02.649155 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.649157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.649170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.649228 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.650122 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.650895 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.651909 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.652211 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.652285 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.655404 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.655405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.655406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.655414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.656755 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.657104 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.657105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.657105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.657153 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.657330 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.657331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.657331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.657450 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.657497 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.657825 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.657911 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:09:02.658187 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.648091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.11.207.131:43439)
0812 21:09:02.648273 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.648361 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.649026 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.649309 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.649314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.649694 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:02.650002 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.650011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.650993 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.650994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.651822 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.651825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.652313 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.652314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.652510 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.653595 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:02.653606 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:02.655296 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:02.656830 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.656835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.656837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.657046 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.657196 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.657196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.657196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.657283 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.657502 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:02.657509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:02.657678 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.657800 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.657978 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260812 21:09:02.658612 26745 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43237
I20260812 21:09:02.658691 26745 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:02.658846 26745 heartbeater.cc:511] Master 127.11.207.190:43237 requested a full tablet report, sending...
I20260812 21:09:02.659243 25869 ts_manager.cc:194] Registered new tserver with Master: 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:02.659713 25869 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:43439
I20260812 21:09:02.664448 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.659633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:49871 (local address 127.11.207.190:43237)
0812 21:09:02.659856 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.659857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.659866 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:02.659884 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.659884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.659894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.659951 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.660244 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.660758 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.661413 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.661611 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.661662 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.662454 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.662455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.662456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.662463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.663466 (+ 1003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.663752 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.663753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.663753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.663809 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.663968 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.663969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.663970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.664072 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.664126 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.664260 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.664361 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260812 21:09:02.664461 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.659590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.11.207.131:49871)
0812 21:09:02.659716 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.659726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.659772 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.659962 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.659963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.660115 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:02.660174 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.660175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.660846 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.660847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.661346 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.661347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.661667 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.661668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.661731 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.662066 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:02.662070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:02.662327 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:02.663535 (+ 1208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.663536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.663536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.663693 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.663861 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:02.663862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:02.663862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:02.663919 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:02.664124 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:02.664125 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:02.664172 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.664233 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.664335 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260812 21:09:02.669358 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:09:02.690615 25958 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:02.685407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.11.207.190:43237)
0812 21:09:02.685473 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.685475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.685485 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:09:02.685554 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.685555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.685564 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.685604 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.686178 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.686664 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.687409 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.687610 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.687668 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.690529 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34710: BlockingRecv error: recv got EOF from 127.0.0.1:34710 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 21:09:02 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:02.697676 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.691332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.11.207.190:43237)
0812 21:09:02.691393 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.691395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.691454 (+ 59us) server_negotiation.cc:407] Connection header received
0812 21:09:02.691519 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.691520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.691529 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.691574 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:02.692039 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.692562 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.693239 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.693431 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.693485 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.695520 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:02.695521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:02.695522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:02.695528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:02.696760 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.697073 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.697073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.697074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.697122 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:02.697335 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:02.697335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:02.697335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:02.697427 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:02.697492 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.697563 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.697616 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:09:02.699177 25869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:09:02.700855 25869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:02.707090 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.703127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.11.207.130:40477)
0812 21:09:02.703326 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.703329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.703551 (+ 222us) server_negotiation.cc:407] Connection header received
0812 21:09:02.703596 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.703598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.703614 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.703690 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:02.703859 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.704980 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706185 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.706678 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706763 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.706826 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.706957 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.706998 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260812 21:09:02.707154 26787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.703248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33559 (local address 127.0.0.1:51544)
0812 21:09:02.703455 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.703464 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.703502 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.703966 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.703967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.703968 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:02.704007 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.704008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.705225 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.705226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.706062 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.706065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.706918 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.706978 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.706982 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.707011 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.707051 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
I20260812 21:09:02.707243 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.703121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.11.207.131:44701)
0812 21:09:02.703277 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.703282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.703468 (+ 186us) server_negotiation.cc:407] Connection header received
0812 21:09:02.703594 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.703597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.703614 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.703668 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:02.703892 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.704651 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706016 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.706764 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706857 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.706918 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.707110 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.707149 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260812 21:09:02.707362 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.703488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.11.207.129:33559)
0812 21:09:02.703653 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:02.703667 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:02.703676 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:02.703712 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:02.703713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:02.703727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:02.704073 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:02.704134 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.705156 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706147 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.706832 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.707130 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.707205 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:02.707230 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 21:09:02.707269 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260812 21:09:02.707067 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.702996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40477 (local address 127.0.0.1:33094)
0812 21:09:02.703352 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.703394 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.703466 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.703699 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.703702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.703704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:02.703769 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.703771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.705080 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.705081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.706114 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.706116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.706752 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.706852 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.706868 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.706935 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.706989 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":10,"thread_start_us":57,"threads_started":1}
I20260812 21:09:02.707762 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:02.703001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44701 (local address 127.0.0.1:42086)
0812 21:09:02.703361 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:02.703398 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:02.703498 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:02.703766 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:02.703767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:02.703768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:02.703806 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.703807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.704728 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.704729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.705932 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:02.705935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:02.706947 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:02.706947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:02.706998 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:02.707001 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:09:02.707041 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 21:09:02.707069 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":88,"threads_started":1}
I20260812 21:09:02.708127 26131 tablet_service.cc:1467] Processing CreateTablet for tablet c7d81f62c1044ba4adbf9a10cd4536e0 (DEFAULT_TABLE table=test-table [id=49e5d9aff95a41afbb01c17bb5dca887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:02.708413 26373 tablet_service.cc:1467] Processing CreateTablet for tablet c7d81f62c1044ba4adbf9a10cd4536e0 (DEFAULT_TABLE table=test-table [id=49e5d9aff95a41afbb01c17bb5dca887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:02.708559 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d81f62c1044ba4adbf9a10cd4536e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:02.708557 26626 tablet_service.cc:1467] Processing CreateTablet for tablet c7d81f62c1044ba4adbf9a10cd4536e0 (DEFAULT_TABLE table=test-table [id=49e5d9aff95a41afbb01c17bb5dca887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:02.708741 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d81f62c1044ba4adbf9a10cd4536e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:02.708901 26626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d81f62c1044ba4adbf9a10cd4536e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:02.711750 26796 tablet_bootstrap.cc:492] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Bootstrap starting.
I20260812 21:09:02.712226 26798 tablet_bootstrap.cc:492] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Bootstrap starting.
I20260812 21:09:02.712527 26796 tablet_bootstrap.cc:654] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:02.712702 26799 tablet_bootstrap.cc:492] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Bootstrap starting.
I20260812 21:09:02.712894 26798 tablet_bootstrap.cc:654] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:02.712911 26796 log.cc:824] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:02.713224 26798 log.cc:824] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:02.713409 26799 tablet_bootstrap.cc:654] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:02.713735 26796 tablet_bootstrap.cc:492] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: No bootstrap required, opened a new log
I20260812 21:09:02.713809 26796 ts_tablet_manager.cc:1397] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:02.713836 26799 log.cc:824] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:02.713958 26798 tablet_bootstrap.cc:492] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: No bootstrap required, opened a new log
I20260812 21:09:02.714031 26798 ts_tablet_manager.cc:1397] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:02.715982 26796 raft_consensus.cc:348] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:02.716164 26796 raft_consensus.cc:374] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:02.716214 26796 raft_consensus.cc:729] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Initialized, Role: FOLLOWER
I20260812 21:09:02.716372 26796 consensus_queue.cc:260] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:02.716321 26798 raft_consensus.cc:348] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:02.716485 26798 raft_consensus.cc:374] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:02.716529 26798 raft_consensus.cc:729] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50030708162549f587620cc6cb921118, State: Initialized, Role: FOLLOWER
I20260812 21:09:02.716657 26796 ts_tablet_manager.cc:1428] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:09:02.716667 26798 consensus_queue.cc:260] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:02.716784 26491 heartbeater.cc:503] Master 127.11.207.190:43237 was elected leader, sending a full tablet report...
I20260812 21:09:02.716929 26798 ts_tablet_manager.cc:1428] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:09:02.717063 26745 heartbeater.cc:503] Master 127.11.207.190:43237 was elected leader, sending a full tablet report...
I20260812 21:09:02.717000 26799 tablet_bootstrap.cc:492] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: No bootstrap required, opened a new log
I20260812 21:09:02.717408 26799 ts_tablet_manager.cc:1397] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 21:09:02.719559 26799 raft_consensus.cc:348] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:02.719702 26799 raft_consensus.cc:374] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:02.719726 26799 raft_consensus.cc:729] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7311a477444d3f9c86d6a12225e9f2, State: Initialized, Role: FOLLOWER
I20260812 21:09:02.719810 26799 consensus_queue.cc:260] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:02.720005 26799 ts_tablet_manager.cc:1428] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:09:02.720062 26247 heartbeater.cc:503] Master 127.11.207.190:43237 was elected leader, sending a full tablet report...
W20260812 21:09:02.741542 26492 tablet.cc:2369] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:09:02.825057 26250 tablet.cc:2369] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:09:02.905752 26746 tablet.cc:2369] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:09:03.020730 26811 raft_consensus.cc:482] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:03.020929 26811 raft_consensus.cc:504] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.021302 26811 leader_election.cc:288] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477), 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:03.024274 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.021623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:48607 (local address 127.11.207.131:44701)
0812 21:09:03.021667 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.021668 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.021745 (+ 77us) server_negotiation.cc:407] Connection header received
0812 21:09:03.021804 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.021804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.021813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.021861 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:03.022008 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.022473 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.023548 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.023961 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.024025 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.024034 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.024166 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.024201 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:09:03.024307 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.021492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40477 (local address 127.11.207.129:42073)
0812 21:09:03.021621 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.021631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.021698 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.021840 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.021841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.021844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:03.021921 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.021924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.022532 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.022532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.023442 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.023445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.024038 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.024039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.024105 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.024109 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.024153 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.024198 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":58,"thread_start_us":60,"threads_started":1}
I20260812 21:09:03.024277 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.021556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:42073 (local address 127.11.207.130:40477)
0812 21:09:03.021626 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.021628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.021695 (+ 67us) server_negotiation.cc:407] Connection header received
0812 21:09:03.021756 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.021757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.021767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.021813 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:03.021980 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.022457 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.023535 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.023966 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.024030 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.024039 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.024168 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.024207 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:09:03.024330 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.021501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44701 (local address 127.11.207.129:48607)
0812 21:09:03.021648 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.021662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.021745 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.021870 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.021871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.021873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:03.021951 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.021953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.022546 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.022547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.023449 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.023452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.024029 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.024030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.024102 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.024106 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.024152 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.024197 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":23,"thread_start_us":56,"threads_started":1}
I20260812 21:09:03.024715 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d81f62c1044ba4adbf9a10cd4536e0" candidate_uuid: "8c7311a477444d3f9c86d6a12225e9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bd7a9f9ee08448994f1488caebe319b" is_pre_election: true
I20260812 21:09:03.024778 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d81f62c1044ba4adbf9a10cd4536e0" candidate_uuid: "8c7311a477444d3f9c86d6a12225e9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50030708162549f587620cc6cb921118" is_pre_election: true
I20260812 21:09:03.024875 26403 raft_consensus.cc:2454] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c7311a477444d3f9c86d6a12225e9f2 in term 0.
I20260812 21:09:03.024948 26658 raft_consensus.cc:2454] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c7311a477444d3f9c86d6a12225e9f2 in term 0.
I20260812 21:09:03.025115 26044 leader_election.cc:302] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c7311a477444d3f9c86d6a12225e9f2, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.025310 26811 raft_consensus.cc:2781] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:03.025372 26811 raft_consensus.cc:482] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:03.025394 26811 raft_consensus.cc:3037] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.026415 26811 raft_consensus.cc:504] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.026546 26811 leader_election.cc:288] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477), 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:03.026693 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d81f62c1044ba4adbf9a10cd4536e0" candidate_uuid: "8c7311a477444d3f9c86d6a12225e9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bd7a9f9ee08448994f1488caebe319b"
I20260812 21:09:03.026734 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d81f62c1044ba4adbf9a10cd4536e0" candidate_uuid: "8c7311a477444d3f9c86d6a12225e9f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50030708162549f587620cc6cb921118"
I20260812 21:09:03.026795 26403 raft_consensus.cc:3037] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.026804 26658 raft_consensus.cc:3037] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.027562 26403 raft_consensus.cc:2454] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c7311a477444d3f9c86d6a12225e9f2 in term 1.
I20260812 21:09:03.027613 26658 raft_consensus.cc:2454] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c7311a477444d3f9c86d6a12225e9f2 in term 1.
I20260812 21:09:03.027730 26044 leader_election.cc:302] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c7311a477444d3f9c86d6a12225e9f2, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.027841 26811 raft_consensus.cc:2781] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:03.027966 26811 raft_consensus.cc:686] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 LEADER]: Becoming Leader. State: Replica: 8c7311a477444d3f9c86d6a12225e9f2, State: Running, Role: LEADER
I20260812 21:09:03.028059 26811 consensus_queue.cc:237] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.028892 25867 catalog_manager.cc:5495] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129). New cstate: current_term: 1 leader_uuid: "8c7311a477444d3f9c86d6a12225e9f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } health_report { overall_health: UNKNOWN } } }
I20260812 21:09:03.063333 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.060864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.11.207.129:33559)
0812 21:09:03.060925 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.060927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.060936 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:03.060978 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.060979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.060998 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.061059 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.061284 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.061766 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.062556 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.062783 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.062849 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.063033 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.063145 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.063209 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.063226 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.063248 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:09:03.065620 26403 raft_consensus.cc:1261] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Refusing update from remote peer 8c7311a477444d3f9c86d6a12225e9f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.065621 26658 raft_consensus.cc:1261] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Refusing update from remote peer 8c7311a477444d3f9c86d6a12225e9f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.065995 26816 consensus_queue.cc:1035] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.066143 26811 consensus_queue.cc:1035] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.068781 26819 mvcc.cc:205] Tried to move back new op lower bound from 7317786390793719808 to 7317786390643499008. Current Snapshot: MvccSnapshot[applied={T|T < 7317786390793719808}]
I20260812 21:09:03.069150 26821 mvcc.cc:205] Tried to move back new op lower bound from 7317786390793719808 to 7317786390643499008. Current Snapshot: MvccSnapshot[applied={T|T < 7317786390793719808}]
I20260812 21:09:03.070127 26820 mvcc.cc:205] Tried to move back new op lower bound from 7317786390793719808 to 7317786390643499008. Current Snapshot: MvccSnapshot[applied={T|T < 7317786390793719808}]
I20260812 21:09:03.071689 25867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.071776 25867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.072566 25867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Sending DeleteTablet for 3 replicas of tablet c7d81f62c1044ba4adbf9a10cd4536e0
I20260812 21:09:03.072949 26626 tablet_service.cc:1514] Processing DeleteTablet for tablet c7d81f62c1044ba4adbf9a10cd4536e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:42086
I20260812 21:09:03.073057 26373 tablet_service.cc:1514] Processing DeleteTablet for tablet c7d81f62c1044ba4adbf9a10cd4536e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:33094
I20260812 21:09:03.073155 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet c7d81f62c1044ba4adbf9a10cd4536e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51544
I20260812 21:09:03.073410 26833 tablet_replica.cc:330] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: stopping tablet replica
I20260812 21:09:03.073484 26830 tablet_replica.cc:330] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: stopping tablet replica
I20260812 21:09:03.073585 26830 raft_consensus.cc:2229] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.073589 26833 raft_consensus.cc:2229] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:09:03.073689 25867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 21:09:03.073848 26830 raft_consensus.cc:2258] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 21:09:03.073861 25867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:03.074084 26832 tablet_replica.cc:330] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: stopping tablet replica
I20260812 21:09:03.073786 26833 raft_consensus.cc:2258] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.074209 26832 raft_consensus.cc:2229] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.074504 26830 ts_tablet_manager.cc:1905] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.074729 26833 ts_tablet_manager.cc:1905] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.075688 26626 tablet_service.cc:1467] Processing CreateTablet for tablet 37b35ae8706847cb87b6f41c09e66a21 (DEFAULT_TABLE table=test-table [id=ffa8646ced7348bb88951162c5c0cea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.075829 26626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b35ae8706847cb87b6f41c09e66a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.076387 26830 ts_tablet_manager.cc:1918] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:03.076468 26830 log.cc:1196] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/c7d81f62c1044ba4adbf9a10cd4536e0
I20260812 21:09:03.076748 26830 ts_tablet_manager.cc:1939] T c7d81f62c1044ba4adbf9a10cd4536e0 P 50030708162549f587620cc6cb921118: Deleting consensus metadata
I20260812 21:09:03.076758 26833 ts_tablet_manager.cc:1918] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:03.076928 26833 log.cc:1196] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/c7d81f62c1044ba4adbf9a10cd4536e0
I20260812 21:09:03.077195 25846 catalog_manager.cc:4841] TS 50030708162549f587620cc6cb921118 (127.11.207.131:44701): tablet c7d81f62c1044ba4adbf9a10cd4536e0 (table test-table [id=49e5d9aff95a41afbb01c17bb5dca887]) successfully deleted
I20260812 21:09:03.077256 26798 tablet_bootstrap.cc:492] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Bootstrap starting.
I20260812 21:09:03.077555 26131 tablet_service.cc:1467] Processing CreateTablet for tablet 37b35ae8706847cb87b6f41c09e66a21 (DEFAULT_TABLE table=test-table [id=ffa8646ced7348bb88951162c5c0cea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.077653 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b35ae8706847cb87b6f41c09e66a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.077730 26798 tablet_bootstrap.cc:654] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.077587 26373 tablet_service.cc:1467] Processing CreateTablet for tablet 37b35ae8706847cb87b6f41c09e66a21 (DEFAULT_TABLE table=test-table [id=ffa8646ced7348bb88951162c5c0cea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.078089 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b35ae8706847cb87b6f41c09e66a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.078378 26833 ts_tablet_manager.cc:1939] T c7d81f62c1044ba4adbf9a10cd4536e0 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting consensus metadata
I20260812 21:09:03.079005 26798 tablet_bootstrap.cc:492] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: No bootstrap required, opened a new log
I20260812 21:09:03.079051 26798 ts_tablet_manager.cc:1397] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:03.079241 26798 raft_consensus.cc:348] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.079308 26798 raft_consensus.cc:374] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.079332 26798 raft_consensus.cc:729] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50030708162549f587620cc6cb921118, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.079378 26798 consensus_queue.cc:260] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.079468 26798 ts_tablet_manager.cc:1428] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:03.079646 26796 tablet_bootstrap.cc:492] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Bootstrap starting.
I20260812 21:09:03.079687 26799 tablet_bootstrap.cc:492] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Bootstrap starting.
I20260812 21:09:03.080117 26796 tablet_bootstrap.cc:654] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.080120 26799 tablet_bootstrap.cc:654] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.080927 26799 tablet_bootstrap.cc:492] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: No bootstrap required, opened a new log
I20260812 21:09:03.080977 26799 ts_tablet_manager.cc:1397] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:03.081058 25845 catalog_manager.cc:4841] TS 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559): tablet c7d81f62c1044ba4adbf9a10cd4536e0 (table test-table [id=49e5d9aff95a41afbb01c17bb5dca887]) successfully deleted
I20260812 21:09:03.081115 26799 raft_consensus.cc:348] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.081183 26799 raft_consensus.cc:374] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.081195 26799 raft_consensus.cc:729] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7311a477444d3f9c86d6a12225e9f2, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.081233 26799 consensus_queue.cc:260] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.081311 26799 ts_tablet_manager.cc:1428] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:03.081624 26832 raft_consensus.cc:2258] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.082115 26832 ts_tablet_manager.cc:1905] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.082252 26796 tablet_bootstrap.cc:492] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: No bootstrap required, opened a new log
I20260812 21:09:03.082314 26796 ts_tablet_manager.cc:1397] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 21:09:03.082499 26796 raft_consensus.cc:348] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.082587 26796 raft_consensus.cc:374] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.082612 26796 raft_consensus.cc:729] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.082679 26796 consensus_queue.cc:260] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.082801 26796 ts_tablet_manager.cc:1428] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:03.082953 26810 raft_consensus.cc:482] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:03.083007 26810 raft_consensus.cc:504] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.083295 26810 leader_election.cc:288] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477)
I20260812 21:09:03.083761 26832 ts_tablet_manager.cc:1918] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:03.083842 26832 log.cc:1196] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/c7d81f62c1044ba4adbf9a10cd4536e0
I20260812 21:09:03.084453 26832 ts_tablet_manager.cc:1939] T c7d81f62c1044ba4adbf9a10cd4536e0 P 9bd7a9f9ee08448994f1488caebe319b: Deleting consensus metadata
I20260812 21:09:03.085227 25845 catalog_manager.cc:4841] TS 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477): tablet c7d81f62c1044ba4adbf9a10cd4536e0 (table test-table [id=49e5d9aff95a41afbb01c17bb5dca887]) successfully deleted
I20260812 21:09:03.087234 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.083593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40477 (local address 127.11.207.131:45745)
0812 21:09:03.083735 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.083747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.083792 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.084129 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.084140 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.084142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:03.084212 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.084214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.085110 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.085110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.086182 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.086185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.086983 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.086985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.087070 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.087074 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.087114 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.087155 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":48,"threads_started":1}
I20260812 21:09:03.087435 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.083931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:45745 (local address 127.11.207.130:40477)
0812 21:09:03.083977 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.083979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.083988 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:03.084011 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.084012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.084021 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.084078 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:03.084293 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.085025 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.086284 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.086858 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.086926 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.086935 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.087332 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.087375 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:09:03.087672 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b35ae8706847cb87b6f41c09e66a21" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bd7a9f9ee08448994f1488caebe319b" is_pre_election: true
I20260812 21:09:03.087764 26403 raft_consensus.cc:2454] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 0.
I20260812 21:09:03.087934 26537 leader_election.cc:302] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.088104 26810 raft_consensus.cc:2781] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:03.088160 26810 raft_consensus.cc:482] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:03.088186 26810 raft_consensus.cc:3037] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.088490 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.083920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:57883 (local address 127.11.207.129:33559)
0812 21:09:03.083963 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.083965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.083972 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:09:03.083995 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.083995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.084003 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.084056 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:03.085588 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.086239 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.087611 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.088081 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.088145 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.088154 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.088382 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.088427 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:09:03.088687 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.083509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33559 (local address 127.11.207.131:57883)
0812 21:09:03.083713 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.083727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.083784 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.085357 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.085358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.085361 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:03.085495 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.085498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.086318 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.086319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.087536 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.087539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.088207 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.088208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.088275 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.088280 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.088567 (+ 287us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.088613 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260812 21:09:03.089226 26810 raft_consensus.cc:504] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.089352 26810 leader_election.cc:288] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 election: Requested vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477)
I20260812 21:09:03.089560 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b35ae8706847cb87b6f41c09e66a21" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bd7a9f9ee08448994f1488caebe319b"
I20260812 21:09:03.089640 26403 raft_consensus.cc:3037] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.089701 26161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b35ae8706847cb87b6f41c09e66a21" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2" is_pre_election: true
I20260812 21:09:03.089846 26161 raft_consensus.cc:2454] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 0.
I20260812 21:09:03.089669 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b35ae8706847cb87b6f41c09e66a21" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2"
I20260812 21:09:03.089980 26160 raft_consensus.cc:3037] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.090381 26403 raft_consensus.cc:2454] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 1.
I20260812 21:09:03.090540 26537 leader_election.cc:302] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.090624 26810 raft_consensus.cc:2781] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:03.090777 26810 raft_consensus.cc:686] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 1 LEADER]: Becoming Leader. State: Replica: 50030708162549f587620cc6cb921118, State: Running, Role: LEADER
I20260812 21:09:03.090864 26810 consensus_queue.cc:237] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.090950 26160 raft_consensus.cc:2454] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 1.
I20260812 21:09:03.091563 25867 catalog_manager.cc:5495] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50030708162549f587620cc6cb921118 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } health_report { overall_health: HEALTHY } } }
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568942, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.190@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:03.121289 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.113437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.11.207.190:43237)
0812 21:09:03.113500 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.113503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.113611 (+ 108us) server_negotiation.cc:407] Connection header received
0812 21:09:03.113663 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.113664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.113678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.113762 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:03.114229 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.115014 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.115944 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.116252 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.116334 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.118729 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:03.118731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:03.118731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:03.118738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:03.120040 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:03.120560 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:03.120561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:03.120562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:03.120624 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:03.120836 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:03.120836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:03.120837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:03.120937 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:03.120984 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.121150 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.121225 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:09:03.121412 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.113341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43237 (local address 127.0.0.1:34722)
0812 21:09:03.113537 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.113547 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.113613 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.113771 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.113772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.114097 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:03.114150 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.114152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.115099 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.115100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.115840 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.115843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.116375 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.116377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.116446 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.117065 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:03.117071 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:03.118642 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:03.120118 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:03.120127 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:03.120130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:03.120453 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:03.120688 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:03.120689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:03.120689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:03.120769 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:03.120991 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:03.120998 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:03.121055 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.121143 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.121302 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":34,"thread_start_us":110,"threads_started":1}
I20260812 21:09:03.123579 25867 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:34722:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 21:09:03.123842 25867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:03.125362 26626 tablet_service.cc:1467] Processing CreateTablet for tablet c6b62b43ed8a4999baf9f821ee9e4747 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=10320bdd460b437a93dc503a4367ddb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:09:03.125469 26131 tablet_service.cc:1467] Processing CreateTablet for tablet c6b62b43ed8a4999baf9f821ee9e4747 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=10320bdd460b437a93dc503a4367ddb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:09:03.125540 26626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b62b43ed8a4999baf9f821ee9e4747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.125497 26373 tablet_service.cc:1467] Processing CreateTablet for tablet c6b62b43ed8a4999baf9f821ee9e4747 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=10320bdd460b437a93dc503a4367ddb0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:09:03.125608 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b62b43ed8a4999baf9f821ee9e4747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.125666 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b62b43ed8a4999baf9f821ee9e4747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.127214 26799 tablet_bootstrap.cc:492] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: Bootstrap starting.
I20260812 21:09:03.127218 26798 tablet_bootstrap.cc:492] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: Bootstrap starting.
I20260812 21:09:03.127355 26796 tablet_bootstrap.cc:492] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: Bootstrap starting.
I20260812 21:09:03.127897 26796 tablet_bootstrap.cc:654] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.127961 26798 tablet_bootstrap.cc:654] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.128014 26799 tablet_bootstrap.cc:654] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.128950 26796 tablet_bootstrap.cc:492] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: No bootstrap required, opened a new log
I20260812 21:09:03.128962 26799 tablet_bootstrap.cc:492] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: No bootstrap required, opened a new log
I20260812 21:09:03.129009 26796 ts_tablet_manager.cc:1397] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:03.129012 26799 ts_tablet_manager.cc:1397] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:09:03.129227 26799 raft_consensus.cc:348] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.129321 26799 raft_consensus.cc:374] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.129331 26798 tablet_bootstrap.cc:492] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: No bootstrap required, opened a new log
I20260812 21:09:03.129346 26799 raft_consensus.cc:729] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7311a477444d3f9c86d6a12225e9f2, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.129375 26798 ts_tablet_manager.cc:1397] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:03.129362 26796 raft_consensus.cc:348] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.129396 26799 consensus_queue.cc:260] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.129433 26796 raft_consensus.cc:374] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.129456 26796 raft_consensus.cc:729] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.129499 26796 consensus_queue.cc:260] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.129513 26799 ts_tablet_manager.cc:1428] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:09:03.129539 26798 raft_consensus.cc:348] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.129590 26796 ts_tablet_manager.cc:1428] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:03.129601 26798 raft_consensus.cc:374] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.129619 26798 raft_consensus.cc:729] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50030708162549f587620cc6cb921118, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.129657 26798 consensus_queue.cc:260] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.129630 26816 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.129704 26816 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.129736 26798 ts_tablet_manager.cc:1428] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:03.129748 26809 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.129895 26809 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.130148 26810 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.130235 26810 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.229303 26810 raft_consensus.cc:482] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:03.229413 26810 raft_consensus.cc:504] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.229581 26810 leader_election.cc:288] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477)
I20260812 21:09:03.229869 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b62b43ed8a4999baf9f821ee9e4747" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2" is_pre_election: true
I20260812 21:09:03.229983 26160 raft_consensus.cc:2454] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 0.
I20260812 21:09:03.229960 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b62b43ed8a4999baf9f821ee9e4747" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bd7a9f9ee08448994f1488caebe319b" is_pre_election: true
I20260812 21:09:03.230088 26403 raft_consensus.cc:2454] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 0.
I20260812 21:09:03.230268 26537 leader_election.cc:302] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 8c7311a477444d3f9c86d6a12225e9f2; no voters:
I20260812 21:09:03.230373 26810 raft_consensus.cc:2781] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:03.230414 26810 raft_consensus.cc:482] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:03.230438 26810 raft_consensus.cc:3037] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.231213 26810 raft_consensus.cc:504] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.231329 26810 leader_election.cc:288] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 election: Requested vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477)
I20260812 21:09:03.231501 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b62b43ed8a4999baf9f821ee9e4747" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2"
I20260812 21:09:03.231516 26403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b62b43ed8a4999baf9f821ee9e4747" candidate_uuid: "50030708162549f587620cc6cb921118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bd7a9f9ee08448994f1488caebe319b"
I20260812 21:09:03.231580 26160 raft_consensus.cc:3037] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.231580 26403 raft_consensus.cc:3037] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.232197 26160 raft_consensus.cc:2454] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 1.
I20260812 21:09:03.232204 26403 raft_consensus.cc:2454] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50030708162549f587620cc6cb921118 in term 1.
I20260812 21:09:03.232398 26537 leader_election.cc:302] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 8c7311a477444d3f9c86d6a12225e9f2; no voters:
I20260812 21:09:03.232494 26810 raft_consensus.cc:2781] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:03.232560 26810 raft_consensus.cc:686] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [term 1 LEADER]: Becoming Leader. State: Replica: 50030708162549f587620cc6cb921118, State: Running, Role: LEADER
I20260812 21:09:03.232611 26810 consensus_queue.cc:237] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.232896 26840 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: TxnStatusTablet state changed. Reason: New leader 50030708162549f587620cc6cb921118. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.232966 26840 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:09:03.233131 26865 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 21:09:03.233253 26865 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 21:09:03.233248 25867 catalog_manager.cc:5495] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50030708162549f587620cc6cb921118 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } health_report { overall_health: HEALTHY } } }
I20260812 21:09:03.247836 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.244996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.11.207.131:44701)
0812 21:09:03.245045 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.245047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.245059 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:09:03.245138 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.245139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.245159 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.245223 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.245384 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.245914 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.247053 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.247250 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.247309 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.247498 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.247597 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.247647 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.247717 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.247742 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:09:03.247844 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.244924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44701 (local address 127.0.0.1:42094)
0812 21:09:03.245000 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.245012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.245063 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.245241 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.245242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.245244 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:09:03.245322 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.245325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.245991 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.245992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.246956 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.246959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.247321 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.247322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.247394 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.247415 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.247650 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.247656 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.247709 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.247758 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:09:03.248989 26403 raft_consensus.cc:1261] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Refusing update from remote peer 50030708162549f587620cc6cb921118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.249141 26160 raft_consensus.cc:1261] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Refusing update from remote peer 50030708162549f587620cc6cb921118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.249423 26840 consensus_queue.cc:1035] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.249567 26810 consensus_queue.cc:1035] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.250197 26809 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: TxnStatusTablet state changed. Reason: New leader 50030708162549f587620cc6cb921118. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.250295 26809 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.250424 26816 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: TxnStatusTablet state changed. Reason: New leader 50030708162549f587620cc6cb921118. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.250512 26816 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.250936 26840 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.251015 26840 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:09:03.251204 26816 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.251279 26816 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 8c7311a477444d3f9c86d6a12225e9f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.251972 26810 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.252051 26810 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 50030708162549f587620cc6cb921118: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:09:03.252614 26809 tablet_replica.cc:439] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50030708162549f587620cc6cb921118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } } }
I20260812 21:09:03.252710 26809 tablet_replica.cc:442] T c6b62b43ed8a4999baf9f821ee9e4747 P 9bd7a9f9ee08448994f1488caebe319b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:09:03.254886 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.252738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.11.207.131:44701)
0812 21:09:03.252777 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.252779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.252900 (+ 121us) server_negotiation.cc:407] Connection header received
0812 21:09:03.252922 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.252923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.252934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.252974 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.253120 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.253594 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.254291 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.254490 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.254542 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.254670 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.254704 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.254747 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.254803 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.254822 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:09:03.260229 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.257730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37919 (local address 127.11.207.131:44701)
0812 21:09:03.257764 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.257766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.258057 (+ 291us) server_negotiation.cc:407] Connection header received
0812 21:09:03.258123 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.258124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.258133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.258172 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.258306 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.258795 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.259625 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.259804 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.259854 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.260008 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.260038 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.260079 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.260130 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.260144 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 21:09:03.260293 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.257682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44701 (local address 127.11.207.131:37919)
0812 21:09:03.257993 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.258003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.258072 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.258183 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.258184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.258185 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:09:03.258259 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.258260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.258865 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.258866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.259552 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.259554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.259858 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.259859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.259925 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.259941 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.260079 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.260081 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.260126 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.260163 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":237,"threads_started":1}
I20260812 21:09:03.262692 26160 raft_consensus.cc:1261] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Refusing update from remote peer 50030708162549f587620cc6cb921118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.262705 26403 raft_consensus.cc:1261] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Refusing update from remote peer 50030708162549f587620cc6cb921118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.262915 26840 consensus_queue.cc:1035] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.263041 26810 consensus_queue.cc:1035] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.277911 25867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.278009 25867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.278573 25867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Sending DeleteTablet for 3 replicas of tablet 37b35ae8706847cb87b6f41c09e66a21
I20260812 21:09:03.278849 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b35ae8706847cb87b6f41c09e66a21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51544
I20260812 21:09:03.278860 26626 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b35ae8706847cb87b6f41c09e66a21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:42086
I20260812 21:09:03.278945 26833 tablet_replica.cc:330] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: stopping tablet replica
I20260812 21:09:03.278967 26373 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b35ae8706847cb87b6f41c09e66a21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:33094
I20260812 21:09:03.279002 26833 raft_consensus.cc:2229] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.279032 26833 raft_consensus.cc:2258] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.279052 26830 tablet_replica.cc:330] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: stopping tablet replica
I20260812 21:09:03.279098 26830 raft_consensus.cc:2229] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:09:03.279178 26830 raft_consensus.cc:2258] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.279250 26832 tablet_replica.cc:330] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: stopping tablet replica
I20260812 21:09:03.279314 26832 raft_consensus.cc:2229] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.279352 26832 raft_consensus.cc:2258] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.279352 26833 ts_tablet_manager.cc:1905] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.279675 26832 ts_tablet_manager.cc:1905] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.279748 26830 ts_tablet_manager.cc:1905] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.281248 26833 ts_tablet_manager.cc:1918] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:09:03.281313 26833 log.cc:1196] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/37b35ae8706847cb87b6f41c09e66a21
I20260812 21:09:03.281392 26830 ts_tablet_manager.cc:1918] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:09:03.281443 26830 log.cc:1196] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/37b35ae8706847cb87b6f41c09e66a21
I20260812 21:09:03.281448 26832 ts_tablet_manager.cc:1918] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:09:03.281498 26832 log.cc:1196] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/37b35ae8706847cb87b6f41c09e66a21
I20260812 21:09:03.281517 26833 ts_tablet_manager.cc:1939] T 37b35ae8706847cb87b6f41c09e66a21 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting consensus metadata
I20260812 21:09:03.281670 26830 ts_tablet_manager.cc:1939] T 37b35ae8706847cb87b6f41c09e66a21 P 50030708162549f587620cc6cb921118: Deleting consensus metadata
I20260812 21:09:03.281706 26832 ts_tablet_manager.cc:1939] T 37b35ae8706847cb87b6f41c09e66a21 P 9bd7a9f9ee08448994f1488caebe319b: Deleting consensus metadata
I20260812 21:09:03.281996 25846 catalog_manager.cc:4841] TS 50030708162549f587620cc6cb921118 (127.11.207.131:44701): tablet 37b35ae8706847cb87b6f41c09e66a21 (table test-table [id=ffa8646ced7348bb88951162c5c0cea3]) successfully deleted
I20260812 21:09:03.282150 25845 catalog_manager.cc:4841] TS 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477): tablet 37b35ae8706847cb87b6f41c09e66a21 (table test-table [id=ffa8646ced7348bb88951162c5c0cea3]) successfully deleted
I20260812 21:09:03.282202 25845 catalog_manager.cc:4841] TS 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559): tablet 37b35ae8706847cb87b6f41c09e66a21 (table test-table [id=ffa8646ced7348bb88951162c5c0cea3]) successfully deleted
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[25784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:09:03.307303 25958 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:03.301557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.11.207.190:43237)
0812 21:09:03.301613 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.301615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.301712 (+ 97us) server_negotiation.cc:407] Connection header received
0812 21:09:03.301752 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.301753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.301765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.301837 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:03.302388 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.303173 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.303882 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.304149 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.304238 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.307231 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34734: BlockingRecv error: recv got EOF from 127.0.0.1:34734 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[25784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM
I20260812 21:09:03.314662 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.308114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.11.207.190:43237)
0812 21:09:03.308169 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.308171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.308180 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:03.308237 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.308238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.308247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.308292 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:03.308759 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.309239 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.309906 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.310091 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.310145 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.312388 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:03.312389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:03.312389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:03.312397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:03.313669 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:03.313973 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:03.313974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:03.313975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:03.314032 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:03.314191 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:03.314192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:03.314192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:03.314304 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:03.314393 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.314489 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.314569 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:09:03.316152 25867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:09:03.316313 25867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:03.317807 26131 tablet_service.cc:1467] Processing CreateTablet for tablet a82a6253ea7b46279bd81b54f65283f0 (DEFAULT_TABLE table=test-table [id=0dbafe0d0cc74b61b02b38320b878b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.317836 26626 tablet_service.cc:1467] Processing CreateTablet for tablet a82a6253ea7b46279bd81b54f65283f0 (DEFAULT_TABLE table=test-table [id=0dbafe0d0cc74b61b02b38320b878b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.317976 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82a6253ea7b46279bd81b54f65283f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.317976 26626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82a6253ea7b46279bd81b54f65283f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.318953 26799 tablet_bootstrap.cc:492] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Bootstrap starting.
I20260812 21:09:03.319263 26799 tablet_bootstrap.cc:654] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.319483 26373 tablet_service.cc:1467] Processing CreateTablet for tablet a82a6253ea7b46279bd81b54f65283f0 (DEFAULT_TABLE table=test-table [id=0dbafe0d0cc74b61b02b38320b878b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.319628 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82a6253ea7b46279bd81b54f65283f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.319885 26799 tablet_bootstrap.cc:492] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: No bootstrap required, opened a new log
I20260812 21:09:03.319931 26799 ts_tablet_manager.cc:1397] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:03.320047 26799 raft_consensus.cc:348] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.320108 26799 raft_consensus.cc:374] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.320127 26799 raft_consensus.cc:729] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7311a477444d3f9c86d6a12225e9f2, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.320168 26799 consensus_queue.cc:260] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.320257 26799 ts_tablet_manager.cc:1428] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:03.320545 26798 tablet_bootstrap.cc:492] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Bootstrap starting.
I20260812 21:09:03.321007 26798 tablet_bootstrap.cc:654] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.321090 26796 tablet_bootstrap.cc:492] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Bootstrap starting.
I20260812 21:09:03.321513 26796 tablet_bootstrap.cc:654] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.322199 26796 tablet_bootstrap.cc:492] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: No bootstrap required, opened a new log
I20260812 21:09:03.322256 26796 ts_tablet_manager.cc:1397] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:03.322275 26798 tablet_bootstrap.cc:492] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: No bootstrap required, opened a new log
I20260812 21:09:03.322333 26798 ts_tablet_manager.cc:1397] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:09:03.322474 26796 raft_consensus.cc:348] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.322503 26798 raft_consensus.cc:348] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.322551 26796 raft_consensus.cc:374] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.322559 26798 raft_consensus.cc:374] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.322573 26796 raft_consensus.cc:729] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.322578 26798 raft_consensus.cc:729] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50030708162549f587620cc6cb921118, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.322620 26796 consensus_queue.cc:260] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.322623 26798 consensus_queue.cc:260] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.322702 26798 ts_tablet_manager.cc:1428] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:03.322702 26796 ts_tablet_manager.cc:1428] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:03.517361 26809 raft_consensus.cc:482] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:03.517490 26809 raft_consensus.cc:504] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.517850 26809 leader_election.cc:288] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:03.521306 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.518165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:46517 (local address 127.11.207.131:44701)
0812 21:09:03.518236 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.518238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.518328 (+ 90us) server_negotiation.cc:407] Connection header received
0812 21:09:03.518403 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.518404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.518416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.518482 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:03.518628 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.519265 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.520494 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.520972 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.521034 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.521043 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.521192 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.521223 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:09:03.521341 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.518062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44701 (local address 127.11.207.130:46517)
0812 21:09:03.518249 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.518260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.518338 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.518489 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.518490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.518492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:03.518558 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.518560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.519368 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.519369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.520382 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.520386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.521056 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.521057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.521119 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.521124 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.521182 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.521245 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260812 21:09:03.521574 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.518262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:48821 (local address 127.11.207.129:33559)
0812 21:09:03.518356 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.518358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.518371 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:09:03.518404 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.518405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.518414 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.518462 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:03.519056 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.519660 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.520781 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.521182 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.521242 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.521251 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.521456 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.521507 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 21:09:03.521639 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a6253ea7b46279bd81b54f65283f0" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50030708162549f587620cc6cb921118" is_pre_election: true
I20260812 21:09:03.521754 26658 raft_consensus.cc:2454] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 0.
I20260812 21:09:03.521813 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.518062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33559 (local address 127.11.207.130:48821)
0812 21:09:03.518240 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.518255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.518342 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.518483 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.518484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.518485 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:03.518955 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.518959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.519794 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.519795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.520666 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.520669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.521257 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.521258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.521343 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.521348 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.521670 (+ 322us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.521719 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":25,"thread_start_us":70,"threads_started":1}
I20260812 21:09:03.521950 26289 leader_election.cc:302] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.522051 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a6253ea7b46279bd81b54f65283f0" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2" is_pre_election: true
I20260812 21:09:03.522095 26809 raft_consensus.cc:2781] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:03.522143 26809 raft_consensus.cc:482] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:03.522146 26160 raft_consensus.cc:2454] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 0.
I20260812 21:09:03.522167 26809 raft_consensus.cc:3037] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.522930 26809 raft_consensus.cc:504] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.523078 26809 leader_election.cc:288] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 election: Requested vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:03.523278 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a6253ea7b46279bd81b54f65283f0" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2"
I20260812 21:09:03.523277 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a6253ea7b46279bd81b54f65283f0" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50030708162549f587620cc6cb921118"
I20260812 21:09:03.523360 26658 raft_consensus.cc:3037] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.523360 26160 raft_consensus.cc:3037] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.524148 26658 raft_consensus.cc:2454] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 1.
I20260812 21:09:03.524148 26160 raft_consensus.cc:2454] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 1.
I20260812 21:09:03.524391 26288 leader_election.cc:302] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c7311a477444d3f9c86d6a12225e9f2, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.524483 26809 raft_consensus.cc:2781] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:03.524675 26809 raft_consensus.cc:686] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 LEADER]: Becoming Leader. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Running, Role: LEADER
I20260812 21:09:03.524760 26809 consensus_queue.cc:237] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.525507 25869 catalog_manager.cc:5495] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130). New cstate: current_term: 1 leader_uuid: "9bd7a9f9ee08448994f1488caebe319b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } health_report { overall_health: UNKNOWN } } }
I20260812 21:09:03.551118 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.548844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.11.207.130:40477)
0812 21:09:03.548909 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.548911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.548950 (+ 39us) server_negotiation.cc:407] Connection header received
0812 21:09:03.549027 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.549028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.549055 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.549117 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.549228 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.549730 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.550449 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.550679 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.550748 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.550848 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.550957 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.551015 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.551030 (+ 15us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.551053 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:09:03.552220 26160 raft_consensus.cc:1261] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Refusing update from remote peer 9bd7a9f9ee08448994f1488caebe319b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.552237 26658 raft_consensus.cc:1261] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Refusing update from remote peer 9bd7a9f9ee08448994f1488caebe319b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.552644 26931 consensus_queue.cc:1035] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.552785 26809 consensus_queue.cc:1035] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [LEADER]: Connected to new peer: Peer: permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.556181 25867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.556290 25867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.556968 25867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Sending DeleteTablet for 3 replicas of tablet a82a6253ea7b46279bd81b54f65283f0
I20260812 21:09:03.557250 26373 tablet_service.cc:1514] Processing DeleteTablet for tablet a82a6253ea7b46279bd81b54f65283f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:33094
I20260812 21:09:03.557241 26626 tablet_service.cc:1514] Processing DeleteTablet for tablet a82a6253ea7b46279bd81b54f65283f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:42086
I20260812 21:09:03.557272 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet a82a6253ea7b46279bd81b54f65283f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51544
I20260812 21:09:03.557343 26832 tablet_replica.cc:330] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: stopping tablet replica
I20260812 21:09:03.557348 26830 tablet_replica.cc:330] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: stopping tablet replica
I20260812 21:09:03.557360 26833 tablet_replica.cc:330] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: stopping tablet replica
I20260812 21:09:03.557390 26832 raft_consensus.cc:2229] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:09:03.557406 26833 raft_consensus.cc:2229] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.557404 26830 raft_consensus.cc:2229] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.557448 26833 raft_consensus.cc:2258] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.557454 26830 raft_consensus.cc:2258] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.557474 26832 raft_consensus.cc:2258] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.557966 26830 ts_tablet_manager.cc:1905] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.558063 25867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:09:03.558212 25867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:03.558497 26833 ts_tablet_manager.cc:1905] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.559132 26832 ts_tablet_manager.cc:1905] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.559777 26830 ts_tablet_manager.cc:1918] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:03.559836 26830 log.cc:1196] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/a82a6253ea7b46279bd81b54f65283f0
I20260812 21:09:03.560021 26833 ts_tablet_manager.cc:1918] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:03.560067 26830 ts_tablet_manager.cc:1939] T a82a6253ea7b46279bd81b54f65283f0 P 50030708162549f587620cc6cb921118: Deleting consensus metadata
I20260812 21:09:03.560074 26833 log.cc:1196] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/a82a6253ea7b46279bd81b54f65283f0
I20260812 21:09:03.560282 26833 ts_tablet_manager.cc:1939] T a82a6253ea7b46279bd81b54f65283f0 P 8c7311a477444d3f9c86d6a12225e9f2: Deleting consensus metadata
I20260812 21:09:03.560461 25846 catalog_manager.cc:4841] TS 50030708162549f587620cc6cb921118 (127.11.207.131:44701): tablet a82a6253ea7b46279bd81b54f65283f0 (table test-table [id=0dbafe0d0cc74b61b02b38320b878b1d]) successfully deleted
I20260812 21:09:03.560734 26832 ts_tablet_manager.cc:1918] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:03.560782 26832 log.cc:1196] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/a82a6253ea7b46279bd81b54f65283f0
I20260812 21:09:03.560968 26832 ts_tablet_manager.cc:1939] T a82a6253ea7b46279bd81b54f65283f0 P 9bd7a9f9ee08448994f1488caebe319b: Deleting consensus metadata
I20260812 21:09:03.561396 25845 catalog_manager.cc:4841] TS 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477): tablet a82a6253ea7b46279bd81b54f65283f0 (table test-table [id=0dbafe0d0cc74b61b02b38320b878b1d]) successfully deleted
I20260812 21:09:03.561386 26626 tablet_service.cc:1467] Processing CreateTablet for tablet 2cff80559ab24ec196ca7b2054ec4787 (DEFAULT_TABLE table=test-table [id=8eb3f39cd5364015b63ac031b1eb9ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.561506 26626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cff80559ab24ec196ca7b2054ec4787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.561574 26131 tablet_service.cc:1467] Processing CreateTablet for tablet 2cff80559ab24ec196ca7b2054ec4787 (DEFAULT_TABLE table=test-table [id=8eb3f39cd5364015b63ac031b1eb9ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.561673 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cff80559ab24ec196ca7b2054ec4787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.562106 25845 catalog_manager.cc:4841] TS 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559): tablet a82a6253ea7b46279bd81b54f65283f0 (table test-table [id=0dbafe0d0cc74b61b02b38320b878b1d]) successfully deleted
I20260812 21:09:03.562727 26798 tablet_bootstrap.cc:492] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Bootstrap starting.
I20260812 21:09:03.562784 26799 tablet_bootstrap.cc:492] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2: Bootstrap starting.
I20260812 21:09:03.562930 26373 tablet_service.cc:1467] Processing CreateTablet for tablet 2cff80559ab24ec196ca7b2054ec4787 (DEFAULT_TABLE table=test-table [id=8eb3f39cd5364015b63ac031b1eb9ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:03.563035 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cff80559ab24ec196ca7b2054ec4787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:03.563062 26798 tablet_bootstrap.cc:654] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.563164 26799 tablet_bootstrap.cc:654] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.563882 26798 tablet_bootstrap.cc:492] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: No bootstrap required, opened a new log
I20260812 21:09:03.563928 26798 ts_tablet_manager.cc:1397] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:03.564046 26798 raft_consensus.cc:348] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.564109 26798 raft_consensus.cc:374] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.564128 26798 raft_consensus.cc:729] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50030708162549f587620cc6cb921118, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.564173 26798 consensus_queue.cc:260] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.564262 26798 ts_tablet_manager.cc:1428] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:03.564500 26796 tablet_bootstrap.cc:492] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Bootstrap starting.
I20260812 21:09:03.564793 26799 tablet_bootstrap.cc:492] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2: No bootstrap required, opened a new log
I20260812 21:09:03.564842 26799 ts_tablet_manager.cc:1397] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:09:03.564944 26799 raft_consensus.cc:348] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.565004 26799 raft_consensus.cc:374] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.565023 26799 raft_consensus.cc:729] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c7311a477444d3f9c86d6a12225e9f2, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.565044 26796 tablet_bootstrap.cc:654] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:03.565063 26799 consensus_queue.cc:260] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.565147 26799 ts_tablet_manager.cc:1428] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:03.565970 26796 tablet_bootstrap.cc:492] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: No bootstrap required, opened a new log
I20260812 21:09:03.566025 26796 ts_tablet_manager.cc:1397] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:09:03.566236 26796 raft_consensus.cc:348] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.566313 26796 raft_consensus.cc:374] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:03.566344 26796 raft_consensus.cc:729] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Initialized, Role: FOLLOWER
I20260812 21:09:03.566399 26796 consensus_queue.cc:260] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.566490 26796 ts_tablet_manager.cc:1428] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:09:03.574898 26944 raft_consensus.cc:482] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:03.574982 26944 raft_consensus.cc:504] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.575162 26944 leader_election.cc:288] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:03.575357 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cff80559ab24ec196ca7b2054ec4787" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50030708162549f587620cc6cb921118" is_pre_election: true
I20260812 21:09:03.575353 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cff80559ab24ec196ca7b2054ec4787" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2" is_pre_election: true
I20260812 21:09:03.575445 26160 raft_consensus.cc:2454] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 0.
I20260812 21:09:03.575449 26658 raft_consensus.cc:2454] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 0.
I20260812 21:09:03.575598 26289 leader_election.cc:302] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.575754 26944 raft_consensus.cc:2781] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:03.575789 26944 raft_consensus.cc:482] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:03.575812 26944 raft_consensus.cc:3037] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.576795 26944 raft_consensus.cc:504] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.576932 26944 leader_election.cc:288] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 election: Requested vote from peers 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559), 50030708162549f587620cc6cb921118 (127.11.207.131:44701)
I20260812 21:09:03.577075 26658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cff80559ab24ec196ca7b2054ec4787" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50030708162549f587620cc6cb921118"
I20260812 21:09:03.577147 26658 raft_consensus.cc:3037] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.577179 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cff80559ab24ec196ca7b2054ec4787" candidate_uuid: "9bd7a9f9ee08448994f1488caebe319b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c7311a477444d3f9c86d6a12225e9f2"
I20260812 21:09:03.577240 26160 raft_consensus.cc:3037] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:03.577952 26160 raft_consensus.cc:2454] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 1.
I20260812 21:09:03.577952 26658 raft_consensus.cc:2454] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bd7a9f9ee08448994f1488caebe319b in term 1.
I20260812 21:09:03.578151 26289 leader_election.cc:302] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50030708162549f587620cc6cb921118, 9bd7a9f9ee08448994f1488caebe319b; no voters:
I20260812 21:09:03.578346 26944 raft_consensus.cc:2781] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:03.578421 26944 raft_consensus.cc:686] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 1 LEADER]: Becoming Leader. State: Replica: 9bd7a9f9ee08448994f1488caebe319b, State: Running, Role: LEADER
I20260812 21:09:03.578505 26944 consensus_queue.cc:237] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } }
I20260812 21:09:03.579035 25867 catalog_manager.cc:5495] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130). New cstate: current_term: 1 leader_uuid: "9bd7a9f9ee08448994f1488caebe319b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bd7a9f9ee08448994f1488caebe319b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 } health_report { overall_health: UNKNOWN } } }
I20260812 21:09:03.592919 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.589833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:46201 (local address 127.11.207.131:44701)
0812 21:09:03.589885 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.589888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.590078 (+ 190us) server_negotiation.cc:407] Connection header received
0812 21:09:03.590162 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.590162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.590183 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.590224 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.590380 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.590854 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.592121 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.592357 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.592413 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.592709 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.592738 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.592777 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.592842 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.592860 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:09:03.593112 26963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.589762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44701 (local address 127.11.207.130:46201)
0812 21:09:03.590006 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.590019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.590103 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.590239 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.590240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.590241 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:09:03.590321 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.590323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.590922 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.590923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.592004 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.592008 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.592431 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.592431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.592596 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.592618 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.592791 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.592792 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.592840 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.592887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":157,"threads_started":1}
I20260812 21:09:03.595341 26658 raft_consensus.cc:1261] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Refusing update from remote peer 9bd7a9f9ee08448994f1488caebe319b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.595443 26160 raft_consensus.cc:1261] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Refusing update from remote peer 9bd7a9f9ee08448994f1488caebe319b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:03.595580 26944 consensus_queue.cc:1035] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [LEADER]: Connected to new peer: Peer: permanent_uuid: "50030708162549f587620cc6cb921118" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.595916 26944 consensus_queue.cc:1035] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c7311a477444d3f9c86d6a12225e9f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:03.603609 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.600813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40477 (local address 127.11.207.131:37911)
0812 21:09:03.600859 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.600869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:03.600903 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:03.601124 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:03.601125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:03.601126 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:09:03.601194 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.601196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.602119 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.602120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.602864 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:03.602867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.603313 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.603314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:03.603356 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.603362 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.603488 (+ 126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.603489 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:09:03.603517 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 21:09:03.603555 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 21:09:03.603623 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.600969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37911 (local address 127.11.207.130:40477)
0812 21:09:03.601014 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:03.601016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:03.601024 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:09:03.601044 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:03.601045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:03.601055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:03.601105 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:03.601257 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.602040 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.602937 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:03.603251 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:03.603316 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:03.603408 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:03.603442 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:03.603487 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:03.603521 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 21:09:03.603540 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:09:03.611936 25869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.612041 25869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:03.612808 25869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077c37c9c2d44c5ab10d323c350d1133: Sending DeleteTablet for 3 replicas of tablet 2cff80559ab24ec196ca7b2054ec4787
I20260812 21:09:03.613080 26626 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cff80559ab24ec196ca7b2054ec4787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:42086
I20260812 21:09:03.613185 26830 tablet_replica.cc:330] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: stopping tablet replica
I20260812 21:09:03.613149 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cff80559ab24ec196ca7b2054ec4787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51544
I20260812 21:09:03.613198 26373 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cff80559ab24ec196ca7b2054ec4787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:03 UTC) from {username='oryx', principal='oryx/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:33094
I20260812 21:09:03.613235 26830 raft_consensus.cc:2229] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.613272 26830 raft_consensus.cc:2258] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.613267 26833 tablet_replica.cc:330] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2: stopping tablet replica
I20260812 21:09:03.613324 26833 raft_consensus.cc:2229] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:03.613360 26833 raft_consensus.cc:2258] T 2cff80559ab24ec196ca7b2054ec4787 P 8c7311a477444d3f9c86d6a12225e9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.613446 26832 tablet_replica.cc:330] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: stopping tablet replica
I20260812 21:09:03.613497 26832 raft_consensus.cc:2229] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:09:03.613560 26832 raft_consensus.cc:2258] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:03.613698 26830 ts_tablet_manager.cc:1905] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.613875 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 26008
I20260812 21:09:03.617687 26830 ts_tablet_manager.cc:1918] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:09:03.617774 26830 log.cc:1196] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/2cff80559ab24ec196ca7b2054ec4787
I20260812 21:09:03.618043 26830 ts_tablet_manager.cc:1939] T 2cff80559ab24ec196ca7b2054ec4787 P 50030708162549f587620cc6cb921118: Deleting consensus metadata
I20260812 21:09:03.618458 25846 catalog_manager.cc:4841] TS 50030708162549f587620cc6cb921118 (127.11.207.131:44701): tablet 2cff80559ab24ec196ca7b2054ec4787 (table test-table [id=8eb3f39cd5364015b63ac031b1eb9ced]) successfully deleted
I20260812 21:09:03.623080 26832 ts_tablet_manager.cc:1905] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:03.624553 26832 ts_tablet_manager.cc:1918] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:09:03.624645 26832 log.cc:1196] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/2cff80559ab24ec196ca7b2054ec4787
I20260812 21:09:03.624861 26832 ts_tablet_manager.cc:1939] T 2cff80559ab24ec196ca7b2054ec4787 P 9bd7a9f9ee08448994f1488caebe319b: Deleting consensus metadata
I20260812 21:09:03.625270 25845 catalog_manager.cc:4841] TS 9bd7a9f9ee08448994f1488caebe319b (127.11.207.130:40477): tablet 2cff80559ab24ec196ca7b2054ec4787 (table test-table [id=8eb3f39cd5364015b63ac031b1eb9ced]) successfully deleted
W20260812 21:09:03.633229 25845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:33559 (error 108)
I20260812 21:09:03.633512 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 26264
I20260812 21:09:03.643832 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:03.633717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33559 (local address 127.0.0.1:51550)
0812 21:09:03.643690 (+ 9973us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:03.643773 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:33559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9945,"thread_start_us":40,"threads_started":1}
W20260812 21:09:03.643990 25845 catalog_manager.cc:4584] TS 8c7311a477444d3f9c86d6a12225e9f2 (127.11.207.129:33559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cff80559ab24ec196ca7b2054ec4787: Network error: Client connection negotiation failed: client connection to 127.11.207.129:33559: connect: Connection refused (error 111)
I20260812 21:09:03.648856 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 26504
I20260812 21:09:03.660557 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 25807
2026-08-12T21:09:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2992 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[26979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[26979](info): set up 2 sockets
Aug 12 21:09:03 dist-test-slave-lkx2 krb5kdc[26979](info): commencing operation
krb5kdc: starting...
W20260812 21:09:04.882733 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:09:04 dist-test-slave-lkx2 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:09:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:09:05.011579 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:33157
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:33157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:05.097359 27006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:05.097587 27006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:05.097612 27006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:05.098754 27006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:05.098795 27006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:05.098810 27006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:05.098824 27006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:05.098838 27006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:05.100437 27006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:33157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:33157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:05.100667 27006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:05.100870 27006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:05.105778 27017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:05.105786 27016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:05.106052 27006 server_base.cc:1034] running on GCE node
W20260812 21:09:05.105785 27024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:05.106389 27006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:05.106693 27006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:05.107847 27006 hybrid_clock.cc:648] HybridClock initialized: now 1786568945107830 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T21:09:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1454 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:09:05 dist-test-slave-lkx2 krb5kdc[27034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:09:05 dist-test-slave-lkx2 krb5kdc[27034](info): set up 2 sockets
Aug 12 21:09:05 dist-test-slave-lkx2 krb5kdc[27034](info): commencing operation
krb5kdc: starting...
W20260812 21:09:06.320637 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:09:06.442891 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38807
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:38807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:06.525894 27059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:06.526108 27059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:06.526131 27059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:06.527251 27059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:06.527287 27059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:06.527299 27059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:06.527310 27059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:06.527320 27059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:06.528836 27059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:38807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:06.529042 27059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:06.529223 27059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:06.533938 27069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:06.533952 27070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:06.534199 27059 server_base.cc:1034] running on GCE node
W20260812 21:09:06.533963 27074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:06.534497 27059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:06.534830 27059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:06.535996 27059 hybrid_clock.cc:648] HybridClock initialized: now 1786568946535974 us; error 36 us; skew 500 ppm
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:06.538717 27059 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:09:06.539162 27059 webserver.cc:460] Webserver started at http://127.11.207.190:42061/ using document root <none> and password file <none>
I20260812 21:09:06.539384 27059 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:06.539435 27059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:06.539578 27059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:06.540747 27059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "24cb7a51079f4b58ac082901137bef99"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "bbb965b7a7d62d4b4e67ac5fb00f874e"
server_key_iv: "eae6a0409c871cc7f59fdbc7289c0bb6"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.541106 27059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "24cb7a51079f4b58ac082901137bef99"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "bbb965b7a7d62d4b4e67ac5fb00f874e"
server_key_iv: "eae6a0409c871cc7f59fdbc7289c0bb6"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.542598 27059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:09:06.543354 27087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.543525 27059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:06.543622 27059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "24cb7a51079f4b58ac082901137bef99"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "bbb965b7a7d62d4b4e67ac5fb00f874e"
server_key_iv: "eae6a0409c871cc7f59fdbc7289c0bb6"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.543717 27059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:06.563771 27059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:06.565726 27059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:06.565878 27059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:06.570186 27059 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:38807
I20260812 21:09:06.570240 27186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:38807 every 8 connection(s)
I20260812 21:09:06.570582 27059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:09:06.571476 27187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:06.573839 27187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Bootstrap starting.
I20260812 21:09:06.574379 27187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:06.574622 27187 log.cc:824] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:06.575255 27187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: No bootstrap required, opened a new log
I20260812 21:09:06.576879 27187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:06.577018 27187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:06.577039 27187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24cb7a51079f4b58ac082901137bef99, State: Initialized, Role: FOLLOWER
I20260812 21:09:06.577153 27187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:06.577231 27187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:09:06.577258 27187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:09:06.577291 27187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:06.577756 27187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:06.577852 27187 leader_election.cc:302] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24cb7a51079f4b58ac082901137bef99; no voters:
I20260812 21:09:06.577988 27187 leader_election.cc:288] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:09:06.578047 27192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:06.578099 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 27059
I20260812 21:09:06.578171 27192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 LEADER]: Becoming Leader. State: Replica: 24cb7a51079f4b58ac082901137bef99, State: Running, Role: LEADER
I20260812 21:09:06.578207 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:09:06.578250 27187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:09:06.578254 27192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:06.578408 12094 external_mini_cluster.cc:1428] Setting key 91934f9d8dfc0761644d86759a25ad64
I20260812 21:09:06.578862 27195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24cb7a51079f4b58ac082901137bef99. Latest consensus state: current_term: 1 leader_uuid: "24cb7a51079f4b58ac082901137bef99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } } }
I20260812 21:09:06.578970 27195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:06.579142 27193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24cb7a51079f4b58ac082901137bef99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } } }
I20260812 21:09:06.579429 27193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:06.579674 27205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:06.580268 27205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:06.581715 27205 catalog_manager.cc:1347] Generated new cluster ID: 9ca4510b01264866ba050efb733ca1b1
I20260812 21:09:06.581768 27205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:06.590128 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:06.579051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.11.207.190:38807)
0812 21:09:06.579204 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:06.579213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:06.579478 (+ 265us) server_negotiation.cc:407] Connection header received
0812 21:09:06.579613 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:06.579621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:06.579683 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:06.579884 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:06.580694 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.582030 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.583090 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.583732 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.583849 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:06.586585 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:06.586589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:06.586590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:06.586602 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:06.588470 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:06.588874 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:06.588877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:06.588877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:06.588994 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:06.589195 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:06.589196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:06.589196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:06.589496 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:06.589633 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:06.589752 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 21:09:06.589888 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":53,"threads_started":1}
I20260812 21:09:06.604389 27205 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:09:06.604969 27205 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:06.607062 27205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Generated new TSK 0
I20260812 21:09:06.607276 27205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:09:06.634795 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38807
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:06.719161 27225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:06.719378 27225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:06.719401 27225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:06.720615 27225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:06.720659 27225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:06.720685 27225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:09:06.722172 27225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:06.722409 27225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:06.722658 27225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:06.727628 27234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:06.727571 27241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:06.727639 27235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:06.727955 27225 server_base.cc:1034] running on GCE node
I20260812 21:09:06.728163 27225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:06.728427 27225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:06.729562 27225 hybrid_clock.cc:648] HybridClock initialized: now 1786568946729554 us; error 28 us; skew 500 ppm
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:06.732373 27225 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:09:06.732939 27225 webserver.cc:460] Webserver started at http://127.11.207.129:46425/ using document root <none> and password file <none>
I20260812 21:09:06.733188 27225 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:06.733235 27225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:06.733363 27225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:06.734485 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "b7fa1670d3af4de2ac17d54271f4f9d5"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "f51582c11d803d17603d0548fe8d8dbd"
server_key_iv: "460eaef70bacfe8fc0939319dcc2f716"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.734839 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "b7fa1670d3af4de2ac17d54271f4f9d5"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "f51582c11d803d17603d0548fe8d8dbd"
server_key_iv: "460eaef70bacfe8fc0939319dcc2f716"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.736153 27225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:06.736953 27255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.737128 27225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.737214 27225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "b7fa1670d3af4de2ac17d54271f4f9d5"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "f51582c11d803d17603d0548fe8d8dbd"
server_key_iv: "460eaef70bacfe8fc0939319dcc2f716"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.737287 27225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:06.761423 27225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:06.763495 27225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:06.763631 27225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:06.763918 27225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:06.764245 27225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:06.764279 27225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.764303 27225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:06.764326 27225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.770081 27225 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:38519
I20260812 21:09:06.770143 27479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:38519 every 8 connection(s)
I20260812 21:09:06.770444 27225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:09:06.771235 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 27225
I20260812 21:09:06.771349 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:09:06.771534 12094 external_mini_cluster.cc:1428] Setting key df3fa8eb37aa173d4a172f62d4a7a797
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:06.780509 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:06.771150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:39393 (local address 127.11.207.190:38807)
0812 21:09:06.771196 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:06.771198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:06.771451 (+ 253us) server_negotiation.cc:407] Connection header received
0812 21:09:06.772079 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:06.772080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:06.772092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:06.772134 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:06.773146 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.773847 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.774876 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.775089 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.775146 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:06.777501 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:06.777502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:06.777503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:06.777511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:06.779062 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:06.779551 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:06.779553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:06.779553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:06.779619 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:06.779850 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:06.779851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:06.779851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:06.779987 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:06.780058 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:06.780298 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 21:09:06.780412 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:09:06.780615 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:06.771135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:39393)
0812 21:09:06.771322 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:06.771372 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:06.771995 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:06.772220 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:06.772224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:06.772855 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:06.773052 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:06.773057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.773941 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.773942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:06.774784 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:06.774786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.775161 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.775162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:06.775303 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:06.775805 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:06.775828 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:06.777406 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:06.779191 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:06.779199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:06.779202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:06.779474 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:06.779697 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:06.779698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:06.779698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:06.779793 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:06.780048 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:06.780052 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:06.780171 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 21:09:06.780255 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:09:06.780422 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260812 21:09:06.781142 27484 heartbeater.cc:347] Connected to a master server at 127.11.207.190:38807
I20260812 21:09:06.781239 27484 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:06.781421 27484 heartbeater.cc:511] Master 127.11.207.190:38807 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
I20260812 21:09:06.781981 27122 ts_manager.cc:194] Registered new tserver with Master: b7fa1670d3af4de2ac17d54271f4f9d5 (127.11.207.129:38519)
I20260812 21:09:06.782603 27122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:39393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:09:06.798930 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38807
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:06.882872 27491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:06.883090 27491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:06.883113 27491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:06.884272 27491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:06.884307 27491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:06.884332 27491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:09:06.885842 27491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:06.886041 27491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:06.886233 27491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:06.891112 27503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:06.891113 27506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:06.891222 27491 server_base.cc:1034] running on GCE node
W20260812 21:09:06.891155 27502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:06.891537 27491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:06.891780 27491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:06.892890 27491 hybrid_clock.cc:648] HybridClock initialized: now 1786568946892875 us; error 31 us; skew 500 ppm
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:06.895319 27491 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:09:06.895749 27491 webserver.cc:460] Webserver started at http://127.11.207.130:39807/ using document root <none> and password file <none>
I20260812 21:09:06.895975 27491 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:06.896026 27491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:06.896150 27491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:06.897257 27491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "c61476fdac484ad784f9a87df1f56e93"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "4a17b6bbd43d44c2f5b32e5cefda35a9"
server_key_iv: "875c4366f1a26d61b7c4461f4e0242ff"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.897589 27491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "c61476fdac484ad784f9a87df1f56e93"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "4a17b6bbd43d44c2f5b32e5cefda35a9"
server_key_iv: "875c4366f1a26d61b7c4461f4e0242ff"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.898821 27491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:09:06.899508 27515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.899657 27491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:06.899750 27491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "c61476fdac484ad784f9a87df1f56e93"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "4a17b6bbd43d44c2f5b32e5cefda35a9"
server_key_iv: "875c4366f1a26d61b7c4461f4e0242ff"
server_key_version: "encryptionkey@0"
I20260812 21:09:06.899818 27491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:06.920562 27491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:06.922401 27491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:06.922545 27491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:06.922801 27491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:06.923115 27491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:06.923147 27491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.923179 27491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:06.923203 27491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:06.929101 27491 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:44539
I20260812 21:09:06.929167 27731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:44539 every 8 connection(s)
I20260812 21:09:06.929478 27491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:09:06.934434 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 27491
I20260812 21:09:06.934525 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:09:06.934686 12094 external_mini_cluster.cc:1428] Setting key 603d9c91fe176ee8df990476c5f01f83
Aug 12 21:09:06 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568946, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:06.938874 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:06.930156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:43167 (local address 127.11.207.190:38807)
0812 21:09:06.930203 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:06.930204 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:06.930413 (+ 209us) server_negotiation.cc:407] Connection header received
0812 21:09:06.930954 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:06.930955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:06.930969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:06.931027 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:06.931959 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.932508 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.933551 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.933763 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.933819 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:06.936106 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:06.936108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:06.936108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:06.936116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:06.937493 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:06.937948 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:06.937949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:06.937949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:06.938003 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:06.938264 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:06.938265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:06.938265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:06.938390 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:06.938453 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:06.938688 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 21:09:06.938775 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:09:06.938953 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:06.930130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:43167)
0812 21:09:06.930291 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:06.930329 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:06.930854 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:06.931092 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:06.931096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:06.931664 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:06.931863 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:06.931869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.932588 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.932589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:06.933471 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:06.933473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:06.933833 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:06.933835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:06.933954 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:06.934432 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:06.934441 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:06.936020 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:06.937573 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:06.937581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:06.937584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:06.937879 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:06.938059 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:06.938060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:06.938060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:06.938204 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:06.938447 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:06.938451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:06.938568 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 21:09:06.938649 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 21:09:06.938776 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260812 21:09:06.939644 27732 heartbeater.cc:347] Connected to a master server at 127.11.207.190:38807
I20260812 21:09:06.939796 27732 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:06.939976 27732 heartbeater.cc:511] Master 127.11.207.190:38807 requested a full tablet report, sending...
I20260812 21:09:06.940450 27122 ts_manager.cc:194] Registered new tserver with Master: c61476fdac484ad784f9a87df1f56e93 (127.11.207.130:44539)
I20260812 21:09:06.940842 27122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:43167
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:09:06.960557 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38807
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:07.044054 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:07.044270 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:07.044292 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:07.045513 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:07.045557 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:07.045583 27739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:09:07.047076 27739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:07.047290 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:07.047497 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:09:07.052631 27739 server_base.cc:1034] running on GCE node
W20260812 21:09:07.052594 27751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.052605 27754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.052573 27752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:07.052973 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:07.053215 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:07.054356 27739 hybrid_clock.cc:648] HybridClock initialized: now 1786568947054331 us; error 38 us; skew 500 ppm
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568947, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:07.056962 27739 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:09:07.057422 27739 webserver.cc:460] Webserver started at http://127.11.207.131:45999/ using document root <none> and password file <none>
I20260812 21:09:07.057653 27739 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:07.057698 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:07.057830 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:07.058866 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "ea145c85ed9b4b13928ad73bf1d2948f"
format_stamp: "Formatted at 2026-08-12 21:09:07 on dist-test-slave-lkx2"
server_key: "15a7e9f5712dfc9d85a29fc684f79a82"
server_key_iv: "07d3415219056937c3acdd3dd174e43c"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.059193 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "ea145c85ed9b4b13928ad73bf1d2948f"
format_stamp: "Formatted at 2026-08-12 21:09:07 on dist-test-slave-lkx2"
server_key: "15a7e9f5712dfc9d85a29fc684f79a82"
server_key_iv: "07d3415219056937c3acdd3dd174e43c"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.060473 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:07.061269 27767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.061470 27739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:07.061549 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ea145c85ed9b4b13928ad73bf1d2948f"
format_stamp: "Formatted at 2026-08-12 21:09:07 on dist-test-slave-lkx2"
server_key: "15a7e9f5712dfc9d85a29fc684f79a82"
server_key_iv: "07d3415219056937c3acdd3dd174e43c"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.061618 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:07.075577 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:07.077646 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:07.077795 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:07.078018 27739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:07.078323 27739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:07.078357 27739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.078382 27739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:07.078405 27739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.084215 27739 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:39645
I20260812 21:09:07.084270 27961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:39645 every 8 connection(s)
I20260812 21:09:07.084601 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:09:07.086017 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 27739
I20260812 21:09:07.086122 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:09:07.086308 12094 external_mini_cluster.cc:1428] Setting key 3f8dc3df5b07d6b7af88b5ecaeddb0a8
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568947, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:07.093734 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.085271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:38847 (local address 127.11.207.190:38807)
0812 21:09:07.085334 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.085336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.085539 (+ 203us) server_negotiation.cc:407] Connection header received
0812 21:09:07.086058 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.086059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.086072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.086125 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.087227 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.087765 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.088803 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.089010 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.089066 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.091146 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:07.091148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:07.091148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:07.091157 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:07.092433 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:07.092803 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:07.092804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:07.092804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:07.092865 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:07.093101 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:07.093102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:07.093102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:07.093217 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:07.093274 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:07.093551 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 21:09:07.093637 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:09:07.093906 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.085256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:38847)
0812 21:09:07.085424 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.085462 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.085957 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.086194 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.086198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.086873 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.087139 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.087146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.087856 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.087857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.088725 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.088728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.089079 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.089080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.089204 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.089638 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.089646 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.091059 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:07.092509 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:07.092516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:07.092518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:07.092735 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:07.092921 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:07.092922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:07.092922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:07.093025 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:07.093274 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:07.093280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:07.093426 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 21:09:07.093519 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 21:09:07.093678 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260812 21:09:07.094390 27962 heartbeater.cc:347] Connected to a master server at 127.11.207.190:38807
I20260812 21:09:07.094486 27962 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:07.094657 27962 heartbeater.cc:511] Master 127.11.207.190:38807 requested a full tablet report, sending...
I20260812 21:09:07.095088 27122 ts_manager.cc:194] Registered new tserver with Master: ea145c85ed9b4b13928ad73bf1d2948f (127.11.207.131:39645)
I20260812 21:09:07.095456 27122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:38847
I20260812 21:09:07.098070 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab.
Entry for principal oryx/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab.
WARNING: no policy specified for oryx/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab.
Entry for principal oryx/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab.
WARNING: no policy specified for oryx/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab.
Entry for principal oryx/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab.
I20260812 21:09:07.168529 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 27225
I20260812 21:09:07.178853 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 27491
I20260812 21:09:07.196410 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 27739
I20260812 21:09:07.208242 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 27059
I20260812 21:09:07.224535 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38807
--webserver_interface=127.11.207.190
--webserver_port=42061
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:38807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:07.307083 27985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:07.307299 27985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:07.307322 27985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:07.308650 27985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:07.308699 27985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:07.308712 27985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:07.308725 27985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:07.308740 27985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:07.310310 27985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:38807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=42061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:07.310544 27985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:07.310756 27985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:07.315500 27996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.315546 27997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.315510 28003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:07.315763 27985 server_base.cc:1034] running on GCE node
I20260812 21:09:07.316005 27985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:07.316269 27985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:07.317413 27985 hybrid_clock.cc:648] HybridClock initialized: now 1786568947317407 us; error 38 us; skew 500 ppm
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568947, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:07.319790 27985 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:09:07.320243 27985 webserver.cc:460] Webserver started at http://127.11.207.190:42061/ using document root <none> and password file <none>
I20260812 21:09:07.320523 27985 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:07.320597 27985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:07.321995 27985 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:07.322660 28015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.322888 27985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:07.322968 27985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "24cb7a51079f4b58ac082901137bef99"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "bbb965b7a7d62d4b4e67ac5fb00f874e"
server_key_iv: "eae6a0409c871cc7f59fdbc7289c0bb6"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.323307 27985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:07.341287 27985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:07.343322 27985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:07.343466 27985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:07.347604 27985 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:38807
I20260812 21:09:07.347669 28128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:38807 every 8 connection(s)
I20260812 21:09:07.348033 27985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:09:07.349593 28129 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:09:07.349627 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 27985
I20260812 21:09:07.349892 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:38519
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:38807
--webserver_port=46425
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:09:07.350340 28129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Bootstrap starting.
I20260812 21:09:07.359773 28129 log.cc:824] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:07.361702 28129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:09:07.361999 28129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Bootstrap complete.
I20260812 21:09:07.364274 28129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:07.364573 28129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24cb7a51079f4b58ac082901137bef99, State: Initialized, Role: FOLLOWER
I20260812 21:09:07.364753 28129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:07.364821 28129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:09:07.364872 28129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:09:07.364924 28129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:09:07.365971 28129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:07.366077 28129 leader_election.cc:302] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24cb7a51079f4b58ac082901137bef99; no voters:
I20260812 21:09:07.366268 28129 leader_election.cc:288] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:09:07.366319 28134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:09:07.366495 28134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [term 2 LEADER]: Becoming Leader. State: Replica: 24cb7a51079f4b58ac082901137bef99, State: Running, Role: LEADER
I20260812 21:09:07.366571 28129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:09:07.366592 28134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } }
I20260812 21:09:07.366888 28134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24cb7a51079f4b58ac082901137bef99. Latest consensus state: current_term: 2 leader_uuid: "24cb7a51079f4b58ac082901137bef99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } } }
I20260812 21:09:07.366967 28134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:07.367112 28137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "24cb7a51079f4b58ac082901137bef99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cb7a51079f4b58ac082901137bef99" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38807 } } }
I20260812 21:09:07.367184 28137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:07.367434 28144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:07.367964 28144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:07.368093 28144 catalog_manager.cc:1259] Loaded cluster ID: 9ca4510b01264866ba050efb733ca1b1
I20260812 21:09:07.368125 28144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:09:07.369565 28144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:07.369752 28144 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 24cb7a51079f4b58ac082901137bef99: Loaded TSK: 0
I20260812 21:09:07.369972 28144 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:09:07.435240 28132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:07.435452 28132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:07.435475 28132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:07.436703 28132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:07.436746 28132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:07.436772 28132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:09:07.438251 28132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.129.keytab
--principal=oryx/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:38519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=46425
--tserver_master_addrs=127.11.207.190:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:07.438491 28132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:07.438720 28132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:07.443804 28170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.443830 28176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.444002 28172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:07.444325 28132 server_base.cc:1034] running on GCE node
I20260812 21:09:07.444545 28132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:07.444777 28132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:07.445924 28132 hybrid_clock.cc:648] HybridClock initialized: now 1786568947445909 us; error 33 us; skew 500 ppm
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568947, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:07.448453 28132 init.cc:377] Logged in from keytab as oryx/127.11.207.129@KRBTEST.COM (short username oryx)
I20260812 21:09:07.448920 28132 webserver.cc:460] Webserver started at http://127.11.207.129:46425/ using document root <none> and password file <none>
I20260812 21:09:07.449162 28132 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:07.449219 28132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:07.450528 28132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:07.451102 28195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.451263 28132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.451362 28132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "b7fa1670d3af4de2ac17d54271f4f9d5"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "f51582c11d803d17603d0548fe8d8dbd"
server_key_iv: "460eaef70bacfe8fc0939319dcc2f716"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.451709 28132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:07.473770 28132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:07.475970 28132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:07.476097 28132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:07.476397 28132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:07.476732 28132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:07.476765 28132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.476788 28132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:07.476811 28132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.483880 28132 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:38519
I20260812 21:09:07.483970 28410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:38519 every 8 connection(s)
I20260812 21:09:07.484303 28132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:09:07.485780 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 28132
I20260812 21:09:07.486071 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:44539
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:38807
--webserver_port=39807
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.493578 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.485088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:49745)
0812 21:09:07.485278 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.485338 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.485807 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.486382 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.486385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.486952 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.487149 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.487155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.488634 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.488635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.489840 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.489844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.490471 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.490473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.490611 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.491096 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.491105 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.493354 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
W20260812 21:09:07.493664 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.485137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:49745 (local address 127.11.207.190:38807)
0812 21:09:07.485367 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.485375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.485417 (+ 42us) server_negotiation.cc:407] Connection header received
0812 21:09:07.486046 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.486052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.486096 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.486226 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.487277 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.488524 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.489938 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.490379 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.490486 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.493512 (+ 3026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:49745: BlockingRecv error: recv got EOF from 127.11.207.129:49745 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
W20260812 21:09:07.493916 28413 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:38807 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.207.190:38807: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.498040 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.494129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:46499)
0812 21:09:07.494178 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.494189 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.494236 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.494364 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.494365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.494490 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.494535 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.494537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.495131 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.495131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.495632 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.495633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.495955 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.495956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.496020 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.496447 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.496452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.497975 (+ 1523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 21:09:07.498119 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.494189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:46499 (local address 127.11.207.190:38807)
0812 21:09:07.494243 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.494245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.494254 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:07.494286 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.494287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.494298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.494346 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.494598 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.495064 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.495701 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.495899 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.495951 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.498069 (+ 2118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:46499: BlockingRecv error: recv got EOF from 127.11.207.129:46499 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.502794 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.498266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:60659)
0812 21:09:07.498395 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.498405 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.498455 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.498576 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.498577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.498703 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.498751 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.498752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.499610 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.499611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.500146 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.500147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.500588 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.500589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.500635 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.501037 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.501042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.502733 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
W20260812 21:09:07.502892 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.498370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:60659 (local address 127.11.207.190:38807)
0812 21:09:07.498409 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.498411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.498465 (+ 54us) server_negotiation.cc:407] Connection header received
0812 21:09:07.498490 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.498491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.498502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.498576 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.498805 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.499540 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.500210 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.500517 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.500577 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.502829 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:60659: BlockingRecv error: recv got EOF from 127.11.207.129:60659 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:09:07.502926 28413 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:09:07.573315 28420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:07.573547 28420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:07.573572 28420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:07.574736 28420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:07.574779 28420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:07.574805 28420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:09:07.576244 28420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.130.keytab
--principal=oryx/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:44539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=39807
--tserver_master_addrs=127.11.207.190:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:07.576483 28420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:07.576705 28420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:07.581590 28440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.581593 28431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:07.581843 28420 server_base.cc:1034] running on GCE node
W20260812 21:09:07.581585 28432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:07.582123 28420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:07.582376 28420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:07.583545 28420 hybrid_clock.cc:648] HybridClock initialized: now 1786568947583529 us; error 29 us; skew 500 ppm
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568947, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:07.586036 28420 init.cc:377] Logged in from keytab as oryx/127.11.207.130@KRBTEST.COM (short username oryx)
I20260812 21:09:07.586479 28420 webserver.cc:460] Webserver started at http://127.11.207.130:39807/ using document root <none> and password file <none>
I20260812 21:09:07.586664 28420 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:07.586716 28420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:07.588001 28420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:07.588595 28453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.588743 28420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:07.588830 28420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "c61476fdac484ad784f9a87df1f56e93"
format_stamp: "Formatted at 2026-08-12 21:09:06 on dist-test-slave-lkx2"
server_key: "4a17b6bbd43d44c2f5b32e5cefda35a9"
server_key_iv: "875c4366f1a26d61b7c4461f4e0242ff"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.589166 28420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:07.596005 28420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:07.598121 28420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:07.598238 28420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:07.598435 28420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:07.598693 28420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:07.598723 28420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.598744 28420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:07.598759 28420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.604450 28420 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:44539
I20260812 21:09:07.604516 28662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:44539 every 8 connection(s)
I20260812 21:09:07.604882 28420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:09:07.612224 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.605624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:38707 (local address 127.11.207.190:38807)
0812 21:09:07.605706 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.605708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.605949 (+ 241us) server_negotiation.cc:407] Connection header received
0812 21:09:07.606438 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.606439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.606455 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.606508 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.607435 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.607935 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.608966 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.609183 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.609256 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.612156 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:38707: BlockingRecv error: recv got EOF from 127.11.207.130:38707 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:09:07.612263 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.605603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:38707)
0812 21:09:07.605826 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.605865 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.606346 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.606564 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.606568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.607143 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.607347 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.607352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.608005 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.608005 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.608867 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.608871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.609258 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.609258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.609358 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.609865 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.609875 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.612049 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":131,"threads_started":1}
W20260812 21:09:07.612550 28664 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:38807 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.207.190:38807: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
I20260812 21:09:07.613756 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 28420
I20260812 21:09:07.614053 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:39645
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:38807
--webserver_port=45999
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.617632 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.612862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:39607)
0812 21:09:07.612912 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.612923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.612967 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.613179 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.613180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.613300 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.613352 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.613354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.614203 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.614204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.614880 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.614884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.615236 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.615237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.615298 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.615712 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.615719 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.617577 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 21:09:07.617831 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.612988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:39607 (local address 127.11.207.190:38807)
0812 21:09:07.613059 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.613061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.613070 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:07.613094 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.613095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.613104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.613156 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.613414 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.614101 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.614967 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.615159 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.615237 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.617769 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:39607: BlockingRecv error: recv got EOF from 127.11.207.130:39607 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.622398 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.618041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:38867)
0812 21:09:07.618087 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.618097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.618129 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.618273 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.618274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.618415 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.618455 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.618456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.619070 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.619072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.619836 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.619839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.620177 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.620178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.620233 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.620638 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.620643 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.622343 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 21:09:07.622553 28664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:09:07.622594 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.618093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:38867 (local address 127.11.207.190:38807)
0812 21:09:07.618131 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.618133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.618184 (+ 51us) server_negotiation.cc:407] Connection header received
0812 21:09:07.618205 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.618206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.618215 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.618253 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.618518 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.618990 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.619924 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.620107 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.620165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.622529 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:38867: BlockingRecv error: recv got EOF from 127.11.207.130:38867 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:09:07.699323 28671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:07.699535 28671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:07.699558 28671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:07.700724 28671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:07.700766 28671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:07.700791 28671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:09:07.702217 28671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/krb5kdc/oryx_127.11.207.131.keytab
--principal=oryx/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:39645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=45999
--tserver_master_addrs=127.11.207.190:38807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:07.702405 28671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:07.702589 28671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:07.707507 28682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.707515 28681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:07.707518 28684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:07.707569 28671 server_base.cc:1034] running on GCE node
I20260812 21:09:07.707834 28671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:07.708062 28671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:07.709221 28671 hybrid_clock.cc:648] HybridClock initialized: now 1786568947709206 us; error 33 us; skew 500 ppm
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568947, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:07.711896 28671 init.cc:377] Logged in from keytab as oryx/127.11.207.131@KRBTEST.COM (short username oryx)
I20260812 21:09:07.712399 28671 webserver.cc:460] Webserver started at http://127.11.207.131:45999/ using document root <none> and password file <none>
I20260812 21:09:07.712613 28671 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:07.712679 28671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:07.714085 28671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:07.714649 28694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.714814 28671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:09:07.714905 28671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ea145c85ed9b4b13928ad73bf1d2948f"
format_stamp: "Formatted at 2026-08-12 21:09:07 on dist-test-slave-lkx2"
server_key: "15a7e9f5712dfc9d85a29fc684f79a82"
server_key_iv: "07d3415219056937c3acdd3dd174e43c"
server_key_version: "encryptionkey@0"
I20260812 21:09:07.715250 28671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:07.724279 28671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:07.726094 28671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:07.726238 28671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:07.726459 28671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:07.726783 28671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:07.726817 28671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.726846 28671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:07.726864 28671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:07.732641 28671 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:39645
I20260812 21:09:07.732712 28941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:39645 every 8 connection(s)
I20260812 21:09:07.733042 28671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:09:07.740027 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.733716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:42869 (local address 127.11.207.190:38807)
0812 21:09:07.733771 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.733773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.734000 (+ 227us) server_negotiation.cc:407] Connection header received
0812 21:09:07.734488 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.734489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.734500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.734539 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.735478 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.735971 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.736962 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.737145 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.737192 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.739978 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:42869: BlockingRecv error: recv got EOF from 127.11.207.131:42869 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:09:07.740047 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.733714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:42869)
0812 21:09:07.733876 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.733914 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.734393 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.734594 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.734597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.735198 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.735399 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.735404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.736054 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.736055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.736898 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.736901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.737205 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.737206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.737312 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.737779 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.737787 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.739897 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
W20260812 21:09:07.740331 28943 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:38807 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.207.190:38807: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
I20260812 21:09:07.740835 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 28671
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.744830 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.740566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:49311)
0812 21:09:07.740608 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.740618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.740655 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.740889 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.740890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.741021 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.741070 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.741071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.741765 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.741766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.742268 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.742269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.742590 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.742590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.742655 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.743045 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.743050 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.744747 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 21:09:07.744899 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.740742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:49311 (local address 127.11.207.190:38807)
0812 21:09:07.740790 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.740792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.740799 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:09:07.740822 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.740823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.740832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.740876 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.741191 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.741695 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.742336 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.742535 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.742587 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.744842 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:49311: BlockingRecv error: recv got EOF from 127.11.207.131:49311 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:09:07.748577 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.741173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.11.207.190:38807)
0812 21:09:07.741307 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.741309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.741351 (+ 42us) server_negotiation.cc:407] Connection header received
0812 21:09:07.741418 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.741418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.741428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.741482 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.742041 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.742502 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.743247 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.743441 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.743493 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.744793 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:07.744798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:07.744799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:07.744812 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:07.746925 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:07.747343 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:07.747346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:07.747347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:07.747480 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:07.747733 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:07.747734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:07.747735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:07.748077 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:07.748238 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:07.748337 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 21:09:07.748475 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:07 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:07.749677 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:07.745073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:58565)
0812 21:09:07.745107 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:07.745116 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:07.745145 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:07.745357 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:07.745358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:07.745505 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:07.745542 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.745544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.746271 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.746272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.747073 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:07.747076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.747476 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.747478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:07.747540 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.747880 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:07.747885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:07.749627 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:09:07.749760 28419 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:07.745154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:58565 (local address 127.11.207.190:38807)
0812 21:09:07.745228 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:07.745230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:07.745238 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:09:07.745256 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:07.745257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:07.745267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:07.745329 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:07.745639 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.746192 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.747146 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:07.747390 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:07.747439 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:07.749707 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:58565: BlockingRecv error: recv got EOF from 127.11.207.131:58565 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 21:09:07.749830 28943 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 21:09:08 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:08 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:08.507937 28951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:08.503343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:49153)
0812 21:09:08.503484 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:08.503495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:08.503558 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:08.503720 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:08.503721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:08.503894 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:08.503948 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:08.503949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.504633 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.504635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:08.505251 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:08.505254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.505621 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.505622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:08.505703 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:08.506110 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:08.506115 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:08.507873 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
W20260812 21:09:08.508044 28955 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:08.503430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:49153 (local address 127.11.207.190:38807)
0812 21:09:08.503560 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:08.503563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:08.503573 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:09:08.503598 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:08.503598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:08.503608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:08.503671 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:08.504008 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.504541 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.505346 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.505548 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.505603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:08.507973 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:49153: BlockingRecv error: recv got EOF from 127.11.207.129:49153 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
Aug 12 21:09:08 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:08 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:08.627509 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:08.622996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:38423)
0812 21:09:08.623137 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:08.623148 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:08.623221 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:08.623361 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:08.623362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:08.623536 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:08.623589 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:08.623590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.624281 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.624282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:08.624944 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:08.624947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.625288 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.625288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:08.625329 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:08.625737 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:08.625742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:08.627430 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
W20260812 21:09:08.627579 28955 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:08.623101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:38423 (local address 127.11.207.190:38807)
0812 21:09:08.623168 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:08.623170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:08.623201 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:09:08.623279 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:08.623280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:08.623291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:08.623351 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:08.623651 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.624197 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.625014 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.625226 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.625276 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:08.627524 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:38423: BlockingRecv error: recv got EOF from 127.11.207.130:38423 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 21:09:08 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:08 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:08.754941 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:08.750258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:38277)
0812 21:09:08.750428 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:08.750441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:08.750511 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:08.750637 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:08.750638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:08.750826 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:08.750879 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:08.750881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.751522 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.751523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:08.752076 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:08.752078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.752435 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.752436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:08.752495 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:08.752938 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:08.752944 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:08.754831 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
W20260812 21:09:08.754979 28955 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:08.750371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:38277 (local address 127.11.207.190:38807)
0812 21:09:08.750432 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:08.750435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:08.750492 (+ 57us) server_negotiation.cc:407] Connection header received
0812 21:09:08.750563 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:08.750564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:08.750577 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:08.750621 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:08.750956 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.751449 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.752154 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:08.752372 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:08.752434 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:08.754930 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:38277: BlockingRecv error: recv got EOF from 127.11.207.131:38277 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 21:09:09 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:09 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:09.512950 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:09.508430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:56181)
0812 21:09:09.508556 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:09.508567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:09.508627 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:09.508788 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:09.508789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:09.508971 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:09.509026 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:09.509028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.509644 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.509645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:09.510193 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:09.510194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.510529 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.510529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:09.510564 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:09.511010 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:09.511015 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:09.512888 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
W20260812 21:09:09.513106 28962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:09.508541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:56181 (local address 127.11.207.190:38807)
0812 21:09:09.508674 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:09.508676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:09.508684 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:09:09.508706 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:09.508707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:09.508718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:09.508774 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:09.509082 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.509590 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.510261 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.510474 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.510525 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:09.513031 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:56181: BlockingRecv error: recv got EOF from 127.11.207.129:56181 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
Aug 12 21:09:09 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:09 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:09.632594 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:09.628022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:37221)
0812 21:09:09.628172 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:09.628185 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:09.628263 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:09.628443 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:09.628444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:09.628622 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:09.628674 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:09.628677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.629328 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.629330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:09.629871 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:09.629873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.630203 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.630204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:09.630257 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:09.630689 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:09.630694 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:09.632532 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
W20260812 21:09:09.632834 28962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:09.628114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:37221 (local address 127.11.207.190:38807)
0812 21:09:09.628186 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:09.628188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:09.628254 (+ 66us) server_negotiation.cc:407] Connection header received
0812 21:09:09.628366 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:09.628367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:09.628378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:09.628426 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:09.628752 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.629249 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.629950 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.630142 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.630199 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:09.632783 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:37221: BlockingRecv error: recv got EOF from 127.11.207.130:37221 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 21:09:09 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:09 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:09.765506 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:09.755377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:58407)
0812 21:09:09.755509 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:09.755520 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:09.755578 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:09.755723 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:09.755723 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:09.755893 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:09.755945 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:09.755947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.756662 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.756664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:09.757196 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:09.757198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.757500 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.757501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:09.757537 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:09.763407 (+ 5870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:09.763413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:09.765430 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
W20260812 21:09:09.765599 28962 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:09.755515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:58407 (local address 127.11.207.190:38807)
0812 21:09:09.755595 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:09.755596 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:09.755605 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:09.755628 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:09.755628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:09.755646 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:09.755712 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:09.756033 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.756592 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.757261 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:09.757445 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:09.757496 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:09.765529 (+ 8033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:58407: BlockingRecv error: recv got EOF from 127.11.207.131:58407 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 21:09:10 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:10 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:10.518162 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:10.513424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:36773)
0812 21:09:10.513579 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:10.513590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:10.513646 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:10.513774 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:10.513775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:10.513938 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:10.513987 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:10.513989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.514610 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.514611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:10.515169 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:10.515171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.515543 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.515544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:10.515613 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:10.516147 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:10.516153 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:10.518079 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
W20260812 21:09:10.518260 28968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:10.513516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:36773 (local address 127.11.207.190:38807)
0812 21:09:10.513624 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:10.513626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:10.513635 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:10.513699 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:10.513700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:10.513710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:10.513762 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:10.514038 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.514542 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.515239 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.515447 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.515497 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:10.518193 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:36773: BlockingRecv error: recv got EOF from 127.11.207.129:36773 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
Aug 12 21:09:10 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:10 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:10.637784 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:10.633136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:44227)
0812 21:09:10.633278 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:10.633291 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:10.633392 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:10.633533 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:10.633534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:10.633719 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:10.633773 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:10.633775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.634433 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.634434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:10.634967 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:10.634969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.635304 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.635305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:10.635371 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:10.635909 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:10.635914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:10.637723 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260812 21:09:10.637892 28968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:10.633224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:44227 (local address 127.11.207.190:38807)
0812 21:09:10.633305 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:10.633307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:10.633370 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:09:10.633460 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:10.633461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:10.633475 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:10.633528 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:10.633861 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.634356 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.635046 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.635243 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.635299 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:10.637828 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:44227: BlockingRecv error: recv got EOF from 127.11.207.130:44227 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 21:09:10 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:10 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:10.771996 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:10.766007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:43589)
0812 21:09:10.767764 (+ 1757us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:10.767776 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:10.767861 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:10.767998 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:10.767999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:10.768170 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:10.768224 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:10.768225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.768935 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.768936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:10.769464 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:10.769466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.769799 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.769800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:10.769847 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:10.770254 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:10.770258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:10.771935 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1735,"thread_start_us":1679,"threads_started":1}
W20260812 21:09:10.772090 28968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:10.766064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:43589 (local address 127.11.207.190:38807)
0812 21:09:10.766127 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:10.766130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:10.767859 (+ 1729us) server_negotiation.cc:407] Connection header received
0812 21:09:10.767904 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:10.767905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:10.767919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:10.767967 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:10.768283 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.768854 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.769531 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:10.769738 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:10.769790 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:10.772029 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:43589: BlockingRecv error: recv got EOF from 127.11.207.131:43589 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 21:09:11 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:11 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:11.523192 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:11.518689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:51549)
0812 21:09:11.518839 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:11.518851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:11.518931 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:11.519057 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:11.519057 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:11.519234 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:11.519286 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:11.519288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.519890 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.519891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:11.520490 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:11.520494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.520854 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.520856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:11.520913 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:11.521420 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:11.521425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:11.523112 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
W20260812 21:09:11.523300 28975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:11.518746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:51549 (local address 127.11.207.190:38807)
0812 21:09:11.518876 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:11.518877 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:11.518924 (+ 47us) server_negotiation.cc:407] Connection header received
0812 21:09:11.518975 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:11.518976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:11.518986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:11.519042 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:11.519336 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.519835 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.520567 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.520770 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.520819 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:11.523224 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:51549: BlockingRecv error: recv got EOF from 127.11.207.129:51549 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
Aug 12 21:09:11 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:11 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:11.642941 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:11.638353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:57807)
0812 21:09:11.638512 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:11.638526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:11.638594 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:11.638771 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:11.638772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:11.638958 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:11.639013 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:11.639015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.639705 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.639706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:11.640245 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:11.640247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.640707 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.640708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:11.640759 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:11.641191 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:11.641202 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:11.642881 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
W20260812 21:09:11.643062 28975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:11.638490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:57807 (local address 127.11.207.190:38807)
0812 21:09:11.638561 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:11.638563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:11.638580 (+ 17us) server_negotiation.cc:407] Connection header received
0812 21:09:11.638652 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:11.638653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:11.638665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:11.638714 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:11.639116 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.639632 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.640396 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.640613 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.640687 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:11.642990 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:57807: BlockingRecv error: recv got EOF from 127.11.207.130:57807 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 21:09:11 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:11 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:11.777437 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:11.772556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:58985)
0812 21:09:11.772754 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:11.772768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:11.772846 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:11.773054 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:11.773056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:11.773230 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:11.773281 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:11.773283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.774136 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.774137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:11.774679 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:11.774681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.774985 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.774986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:11.775031 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:11.775480 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:11.775484 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:11.777348 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
W20260812 21:09:11.777486 28975 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:11.772678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:58985 (local address 127.11.207.190:38807)
0812 21:09:11.772751 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:11.772752 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:11.772837 (+ 85us) server_negotiation.cc:407] Connection header received
0812 21:09:11.772944 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:11.772945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:11.772957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:11.773009 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:11.773366 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.774063 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.774735 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:11.774926 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:11.774971 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:11.777433 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:58985: BlockingRecv error: recv got EOF from 127.11.207.131:58985 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 21:09:12 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:12 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:12.528448 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:12.523666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:49487)
0812 21:09:12.523834 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:12.523846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:12.523904 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:12.524052 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:12.524053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:12.524226 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:12.524279 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:12.524281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.524972 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.524973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:12.525568 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:12.525571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.525934 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.525935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:12.525984 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:12.526451 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:12.526457 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:12.528365 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":65,"threads_started":1}
W20260812 21:09:12.528530 28980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:12.523828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:49487 (local address 127.11.207.190:38807)
0812 21:09:12.523937 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:12.523940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:12.523950 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:09:12.523973 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:12.523973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:12.523983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:12.524038 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:12.524332 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.524878 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.525654 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.525871 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.525924 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:12.528468 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:49487: BlockingRecv error: recv got EOF from 127.11.207.129:49487 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
Aug 12 21:09:12 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:12 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:12.647919 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:12.643446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:58983)
0812 21:09:12.643611 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:12.643622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:12.643678 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:12.643834 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:12.643835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:12.644003 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:12.644056 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:12.644058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.644718 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.644720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:12.645322 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:12.645325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.645657 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.645658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:12.645706 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:12.646120 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:12.646124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:12.647855 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
W20260812 21:09:12.648002 28980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:12.643574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:58983 (local address 127.11.207.190:38807)
0812 21:09:12.643619 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:12.643621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:12.643663 (+ 42us) server_negotiation.cc:407] Connection header received
0812 21:09:12.643741 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:12.643742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:12.643751 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:12.643817 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:12.644110 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.644624 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.645404 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.645598 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.645652 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:12.647950 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:58983: BlockingRecv error: recv got EOF from 127.11.207.130:58983 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 21:09:12 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:12 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:12.782594 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:12.777948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:34395)
0812 21:09:12.778074 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:12.778087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:12.778161 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:12.778299 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:12.778300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:12.778481 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:12.778533 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:12.778535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.779202 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.779203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:12.779762 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:12.779764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.780108 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.780109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:12.780167 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:12.780621 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:12.780626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:12.782516 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
W20260812 21:09:12.782665 28980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:12.778046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:34395 (local address 127.11.207.190:38807)
0812 21:09:12.778124 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:12.778126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:12.778140 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:09:12.778220 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:12.778221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:12.778236 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:12.778282 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:12.778603 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.779107 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.779849 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:12.780045 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:12.780102 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:12.782609 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:34395: BlockingRecv error: recv got EOF from 127.11.207.131:34395 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 21:09:13 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:13 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:13.533411 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:13.528945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:34087)
0812 21:09:13.529094 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:13.529104 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:13.529171 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:13.529310 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:13.529310 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:13.529483 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:13.529537 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:13.529538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.530182 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.530183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:13.530731 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:13.530733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.531087 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.531089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:13.531165 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:13.531602 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:13.531608 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:13.533336 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
W20260812 21:09:13.533530 28985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:13.529006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:34087 (local address 127.11.207.190:38807)
0812 21:09:13.529120 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:13.529122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:13.529148 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:09:13.529221 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:13.529221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:13.529232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:13.529296 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:13.529586 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.530088 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.530801 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.531000 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.531053 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:13.533450 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:34087: BlockingRecv error: recv got EOF from 127.11.207.129:34087 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
Aug 12 21:09:13 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:13 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:13.652971 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:13.648457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:54115)
0812 21:09:13.648605 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:13.648618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:13.648693 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:13.648840 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:13.648841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:13.649026 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:13.649077 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:13.649080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.649736 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.649738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:13.650271 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:13.650272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.650617 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.650618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:13.650674 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:13.651105 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:13.651109 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:13.652889 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
W20260812 21:09:13.653122 28985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:13.648538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:54115 (local address 127.11.207.190:38807)
0812 21:09:13.648609 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:13.648611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:13.648678 (+ 67us) server_negotiation.cc:407] Connection header received
0812 21:09:13.648757 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:13.648758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:13.648771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:13.648827 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:13.649151 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.649658 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.650363 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.650558 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.650611 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:13.652995 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:54115: BlockingRecv error: recv got EOF from 127.11.207.130:54115 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 21:09:13 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:13 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:13.788178 28988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:13.783081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:56733)
0812 21:09:13.783297 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:13.783308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:13.783375 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:13.783512 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:13.783513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:13.783768 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:13.783838 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:13.783842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.784708 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.784710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:13.785318 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:13.785320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.785692 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.785693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:13.785750 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:13.786299 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:13.786304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:13.788116 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
W20260812 21:09:13.788304 28985 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:13.783165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:56733 (local address 127.11.207.190:38807)
0812 21:09:13.783240 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:13.783242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:13.783360 (+ 118us) server_negotiation.cc:407] Connection header received
0812 21:09:13.783437 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:13.783439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:13.783454 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:13.783495 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:13.783952 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.784604 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.785392 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:13.785591 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:13.785652 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:13.788236 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:56733: BlockingRecv error: recv got EOF from 127.11.207.131:56733 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 21:09:14 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:14 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:14.538908 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:14.533950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:46985)
0812 21:09:14.534116 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:14.534127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:14.534194 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:14.534377 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:14.534378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:14.534545 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:14.534604 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:14.534606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.535492 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.535494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:14.536095 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:14.536099 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.536534 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.536536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:14.536590 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:14.537068 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:14.537074 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:14.538823 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
W20260812 21:09:14.539037 28991 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:14.534020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:46985 (local address 127.11.207.190:38807)
0812 21:09:14.534140 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:14.534142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:14.534178 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:09:14.534260 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:14.534260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:14.534271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:14.534351 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:14.534756 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.535385 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.536175 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.536438 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.536497 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:14.538957 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:46985: BlockingRecv error: recv got EOF from 127.11.207.129:46985 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
Aug 12 21:09:14 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:14 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:14.658174 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:14.653484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:37657)
0812 21:09:14.653652 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:14.653664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:14.653759 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:14.653926 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:14.653927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:14.654105 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:14.654157 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:14.654159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.654914 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.654915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:14.655501 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:14.655504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.655837 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.655838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:14.655890 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:14.656365 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:14.656370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:14.658095 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
W20260812 21:09:14.658248 28991 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:14.653582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:37657 (local address 127.11.207.190:38807)
0812 21:09:14.653656 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:14.653659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:14.653725 (+ 66us) server_negotiation.cc:407] Connection header received
0812 21:09:14.653853 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:14.653854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:14.653865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:14.653909 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:14.654226 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.654834 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.655574 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.655775 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.655838 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:14.658197 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:37657: BlockingRecv error: recv got EOF from 127.11.207.130:37657 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 21:09:14 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:14 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:14.792887 28995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:14.788588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:54325)
0812 21:09:14.788733 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:14.788744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:14.788803 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:14.788916 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:14.788917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:14.789074 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:14.789128 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:14.789129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.789705 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.789705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:14.790224 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:14.790225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.790576 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.790577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:14.790613 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:14.791026 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:14.791030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:14.792826 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
W20260812 21:09:14.792980 28991 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:14.788653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:54325 (local address 127.11.207.190:38807)
0812 21:09:14.788713 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:14.788715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:14.788794 (+ 79us) server_negotiation.cc:407] Connection header received
0812 21:09:14.788845 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:14.788845 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:14.788857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:14.788904 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:14.789173 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.789654 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.790309 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:14.790509 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:14.790579 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:14.792921 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:54325: BlockingRecv error: recv got EOF from 127.11.207.131:54325 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 21:09:15 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:15 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:15.547986 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:15.539484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:43029)
0812 21:09:15.539631 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:15.539641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:15.539695 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:15.539903 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:15.539904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:15.540079 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:15.540137 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:15.540139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.540856 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.540857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:15.541386 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:15.541388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.541773 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.541774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:15.541840 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:15.546122 (+ 4282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:15.546128 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:15.547924 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
W20260812 21:09:15.548120 29004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:15.539615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:43029 (local address 127.11.207.190:38807)
0812 21:09:15.539769 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:15.539771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:15.539779 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:09:15.539804 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:15.539804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:15.539814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:15.539881 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:15.540216 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.540791 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.541464 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.541702 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.541781 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:15.548037 (+ 6256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:43029: BlockingRecv error: recv got EOF from 127.11.207.129:43029 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
Aug 12 21:09:15 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:15 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:15.663589 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:15.658699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:33595)
0812 21:09:15.658873 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:15.658886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:15.658965 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:15.659111 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:15.659112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:15.659285 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:15.659340 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:15.659342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.660129 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.660130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:15.660723 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:15.660726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.661078 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.661079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:15.661286 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:15.661778 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:15.661783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:15.663508 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
W20260812 21:09:15.663676 29004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:15.658814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:33595 (local address 127.11.207.190:38807)
0812 21:09:15.658873 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:15.658876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:15.658951 (+ 75us) server_negotiation.cc:407] Connection header received
0812 21:09:15.659016 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:15.659016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:15.659027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:15.659076 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:15.659402 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.660052 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.660809 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.661013 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.661072 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:15.663622 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:33595: BlockingRecv error: recv got EOF from 127.11.207.130:33595 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:09:15 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:15 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:15.797675 29006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:15.793301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:37091)
0812 21:09:15.793442 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:15.793454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:15.793515 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:15.793633 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:15.793634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:15.793804 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:15.793855 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:15.793856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.794503 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.794503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:15.795023 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:15.795025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.795324 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.795324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:15.795361 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:15.795783 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:15.795787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:15.797603 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
W20260812 21:09:15.797768 29004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:15.793354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37091 (local address 127.11.207.190:38807)
0812 21:09:15.793432 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:15.793435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:15.793509 (+ 74us) server_negotiation.cc:407] Connection header received
0812 21:09:15.793562 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:15.793563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:15.793576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:15.793623 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:15.793938 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.794437 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.795077 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:15.795273 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:15.795321 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:15.797707 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:37091: BlockingRecv error: recv got EOF from 127.11.207.131:37091 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 21:09:16 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:16 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:16.553242 29008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:16.548503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:47205)
0812 21:09:16.548646 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:16.548664 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:16.548724 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:16.548855 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:16.548855 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:16.549026 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:16.549079 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:16.549081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.549855 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.549856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:16.550405 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:16.550406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.550747 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.550747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:16.550783 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:16.551236 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:16.551241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:16.553182 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260812 21:09:16.553411 29009 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:16.548593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:47205 (local address 127.11.207.190:38807)
0812 21:09:16.548705 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:16.548706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:16.548717 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:09:16.548783 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:16.548784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:16.548795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:16.548844 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:16.549134 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.549771 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.550476 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.550692 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.550742 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:16.553301 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:47205: BlockingRecv error: recv got EOF from 127.11.207.129:47205 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
Aug 12 21:09:16 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:16 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:09:16.668890 29009 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:16.664195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:57189 (local address 127.11.207.190:38807)
0812 21:09:16.664263 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:16.664265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:16.664290 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:09:16.664398 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:16.664399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:16.664412 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:16.664461 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:16.664787 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.665293 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.665990 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.666186 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.666242 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:16.668836 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:57189: BlockingRecv error: recv got EOF from 127.11.207.130:57189 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:09:16.668915 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:16.664086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:57189)
0812 21:09:16.664227 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:16.664243 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:16.664310 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:16.664474 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:16.664475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:16.664664 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:16.664713 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:16.664715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.665385 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.665387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:16.665916 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:16.665917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.666249 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.666250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:16.666307 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:16.666781 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:16.666787 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:16.668719 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
Aug 12 21:09:16 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:16 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:16.803088 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:16.798204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:53823)
0812 21:09:16.798360 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:16.798371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:16.798438 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:16.798566 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:16.798567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:16.798775 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:16.798831 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:16.798833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.799547 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.799549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:16.800130 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:16.800132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.800537 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.800538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:16.800587 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:16.801094 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:16.801103 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:16.803022 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260812 21:09:16.803202 29009 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:16.798301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:53823 (local address 127.11.207.190:38807)
0812 21:09:16.798358 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:16.798360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:16.798436 (+ 76us) server_negotiation.cc:407] Connection header received
0812 21:09:16.798483 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:16.798484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:16.798499 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:16.798554 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:16.798914 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.799454 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.800227 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:16.800469 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:16.800534 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:16.803134 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:53823: BlockingRecv error: recv got EOF from 127.11.207.131:53823 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 21:09:17 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:17 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:17.558249 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:17.553760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:43589)
0812 21:09:17.553882 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:17.553894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:17.553944 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:17.554070 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:17.554070 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:17.554248 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:17.554301 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:17.554303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.555039 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.555040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:17.555580 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:17.555581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.555897 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.555898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:17.555939 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:17.556371 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:17.556378 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:17.558161 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
W20260812 21:09:17.558338 29014 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:17.553831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:43589 (local address 127.11.207.190:38807)
0812 21:09:17.553945 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:17.553946 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:17.553955 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:17.553981 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:17.553982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:17.553995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:17.554060 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:17.554394 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.554962 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.555639 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.555836 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.555892 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:17.558275 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:43589: BlockingRecv error: recv got EOF from 127.11.207.129:43589 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
Aug 12 21:09:17 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:17 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:17.674443 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:17.669491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:47691)
0812 21:09:17.669638 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:17.669650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:17.669721 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:17.669853 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:17.669854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:17.670031 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:17.670084 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:17.670087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.670897 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.670899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:17.671448 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:17.671450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.671846 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.671848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:17.671886 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:17.672427 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:17.672432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:17.674381 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
W20260812 21:09:17.674571 29014 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:17.669568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:47691 (local address 127.11.207.190:38807)
0812 21:09:17.669634 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:17.669636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:17.669715 (+ 79us) server_negotiation.cc:407] Connection header received
0812 21:09:17.669775 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:17.669776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:17.669788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:17.669848 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:17.670171 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.670814 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.671554 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.671771 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.671837 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:17.674506 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:47691: BlockingRecv error: recv got EOF from 127.11.207.130:47691 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:09:17 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:17 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:17.808081 29025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:17.803553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:37175)
0812 21:09:17.803689 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:17.803703 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:17.803773 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:17.803907 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:17.803908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:17.804110 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:17.804162 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:17.804164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.804870 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.804871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:17.805408 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:17.805410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.805735 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.805736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:17.805794 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:17.806246 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:17.806251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:17.807990 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
W20260812 21:09:17.808142 29014 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:17.803691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37175 (local address 127.11.207.190:38807)
0812 21:09:17.803782 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:17.803784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:17.803811 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:09:17.803835 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:17.803836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:17.803848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:17.803892 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:17.804237 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.804786 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.805485 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:17.805674 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:17.805736 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:17.808088 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:37175: BlockingRecv error: recv got EOF from 127.11.207.131:37175 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 21:09:18 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:18 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:18.563419 29026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:18.558722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:56073)
0812 21:09:18.558860 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:18.558871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:18.558924 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:18.559133 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:18.559134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:18.559342 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:18.559408 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:18.559411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.560230 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.560231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:18.560846 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:18.560849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.561202 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.561202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:18.561241 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:18.561653 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:18.561657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:18.563345 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
W20260812 21:09:18.563522 29028 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:18.558837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:56073 (local address 127.11.207.190:38807)
0812 21:09:18.558964 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:18.558966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:18.558975 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:18.559002 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:18.559003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:18.559015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:18.559121 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:18.559488 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.560142 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.560932 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.561133 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.561200 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:18.563453 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:56073: BlockingRecv error: recv got EOF from 127.11.207.129:56073 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
Aug 12 21:09:18 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:18 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:18.679754 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:18.674978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:48489)
0812 21:09:18.675108 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:18.675118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:18.675188 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:18.675324 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:18.675324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:18.675498 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:18.675549 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:18.675551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.676198 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.676200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:18.676813 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:18.676815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.677194 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.677195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:18.677243 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:18.677646 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:18.677650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:18.679653 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260812 21:09:18.679845 29028 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:18.675060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:48489 (local address 127.11.207.190:38807)
0812 21:09:18.675133 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:18.675135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:18.675170 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:09:18.675241 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:18.675242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:18.675253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:18.675305 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:18.675601 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.676116 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.676910 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.677117 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.677175 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:18.679767 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:48489: BlockingRecv error: recv got EOF from 127.11.207.130:48489 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 21:09:18 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:18 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:18.813143 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:18.808645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:35821)
0812 21:09:18.808829 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:18.808842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:18.808912 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:18.809047 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:18.809048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:18.809282 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:18.809335 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:18.809338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.809972 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.809973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:18.810517 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:18.810519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.810821 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.810821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:18.810858 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:18.811304 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:18.811310 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:18.813072 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
W20260812 21:09:18.813246 29028 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:18.808752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:35821 (local address 127.11.207.190:38807)
0812 21:09:18.808823 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:18.808825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:18.808896 (+ 71us) server_negotiation.cc:407] Connection header received
0812 21:09:18.808968 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:18.808969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:18.808985 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:18.809038 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:18.809403 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.809908 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.810579 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:18.810763 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:18.810810 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:18.813179 (+ 2369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:35821: BlockingRecv error: recv got EOF from 127.11.207.131:35821 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 21:09:19 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:19 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:19.568528 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:19.563937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:42689)
0812 21:09:19.564070 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:19.564081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:19.564142 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:19.564291 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:19.564292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:19.564508 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:19.564559 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:19.564561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.565198 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.565198 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:19.565725 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:19.565727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.566073 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.566074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:19.566124 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:19.566575 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:19.566580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:19.568446 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
W20260812 21:09:19.568612 29033 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:19.564039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:42689 (local address 127.11.207.190:38807)
0812 21:09:19.564142 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:19.564144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:19.564194 (+ 50us) server_negotiation.cc:407] Connection header received
0812 21:09:19.564217 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:19.564218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:19.564230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:19.564278 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:19.564621 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.565123 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.565780 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.566004 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.566054 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:19.568550 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:42689: BlockingRecv error: recv got EOF from 127.11.207.129:42689 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
Aug 12 21:09:19 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:19 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:19.684942 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:19.680252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:57343)
0812 21:09:19.680407 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:19.680419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:19.680476 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:19.680603 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:19.680604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:19.680815 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:19.680866 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:19.680869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.681488 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.681488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:19.682073 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:19.682075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.682372 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.682374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:19.682425 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:19.683020 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:19.683025 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:19.684880 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
W20260812 21:09:19.685055 29033 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:19.680356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:57343 (local address 127.11.207.190:38807)
0812 21:09:19.680414 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:19.680416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:19.680463 (+ 47us) server_negotiation.cc:407] Connection header received
0812 21:09:19.680536 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:19.680537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:19.680549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:19.680588 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:19.680921 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.681419 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.682136 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.682312 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.682354 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:19.684991 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:57343: BlockingRecv error: recv got EOF from 127.11.207.130:57343 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 21:09:19 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:19 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:19.818332 29035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:19.813662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:53707)
0812 21:09:19.813847 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:19.813860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:19.813933 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:19.814059 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:19.814060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:19.814297 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:19.814359 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:19.814361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.815027 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.815028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:19.815579 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:19.815581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.815917 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.815919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:19.815977 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:19.816435 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:19.816441 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:19.818270 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
W20260812 21:09:19.818477 29033 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:19.813777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:53707 (local address 127.11.207.190:38807)
0812 21:09:19.813853 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:19.813855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:19.813917 (+ 62us) server_negotiation.cc:407] Connection header received
0812 21:09:19.813989 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:19.813990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:19.814005 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:19.814054 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:19.814429 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.814951 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.815657 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:19.815855 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:19.815912 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:19.818364 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:53707: BlockingRecv error: recv got EOF from 127.11.207.131:53707 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 21:09:20 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:20 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:20.579669 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:20.569034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:47263)
0812 21:09:20.569221 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:20.569235 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:20.569336 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:20.569534 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:20.569534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:20.569772 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:20.569851 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:20.569854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.570541 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.570542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:20.571183 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:20.571186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.571558 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.571559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:20.571611 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:20.577507 (+ 5896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:20.577514 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:20.579605 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
W20260812 21:09:20.579845 29041 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:20.569092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:47263 (local address 127.11.207.190:38807)
0812 21:09:20.569212 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:20.569214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:20.569323 (+ 109us) server_negotiation.cc:407] Connection header received
0812 21:09:20.569404 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:20.569404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:20.569419 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:20.569511 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:20.569933 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.570463 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.571269 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.571471 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.571527 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:20.579752 (+ 8225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:47263: BlockingRecv error: recv got EOF from 127.11.207.129:47263 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
Aug 12 21:09:20 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:20 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:20.690376 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:20.685547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:46469)
0812 21:09:20.685717 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:20.685731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:20.685801 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:20.685934 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:20.685934 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:20.686115 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:20.686167 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:20.686169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.686868 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.686869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:20.687396 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:20.687398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.687705 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.687705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:20.687743 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:20.688219 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:20.688224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:20.690289 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
W20260812 21:09:20.690444 29041 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:20.685687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:46469 (local address 127.11.207.190:38807)
0812 21:09:20.685804 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:20.685806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:20.685819 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:09:20.685853 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:20.685854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:20.685865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:20.685918 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:20.686271 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.686793 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.687458 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.687653 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.687699 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:20.690393 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:46469: BlockingRecv error: recv got EOF from 127.11.207.130:46469 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 21:09:20 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:20 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:20.823740 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:20.818847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:47153)
0812 21:09:20.819015 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:20.819025 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:20.819091 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:20.819232 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:20.819233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:20.819410 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:20.819461 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:20.819462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.820081 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.820082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:20.820741 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:20.820744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.821210 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.821211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:20.821260 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:20.821709 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:20.821714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:20.823662 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
W20260812 21:09:20.823961 29041 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:20.818964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:47153 (local address 127.11.207.190:38807)
0812 21:09:20.819026 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:20.819027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:20.819085 (+ 58us) server_negotiation.cc:407] Connection header received
0812 21:09:20.819141 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:20.819142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:20.819155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:20.819217 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:20.819515 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.820015 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.820818 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:20.821123 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:20.821182 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:20.823902 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:47153: BlockingRecv error: recv got EOF from 127.11.207.131:47153 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 21:09:21 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:21 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:21.585438 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:21.580237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:56645)
0812 21:09:21.580417 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:21.580432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:21.580503 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:21.580737 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:21.580738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:21.580943 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:21.581002 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:21.581004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.581764 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.581765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:21.582345 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:21.582348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.582788 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.582789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:21.582856 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:21.583354 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:21.583360 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:21.585373 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
W20260812 21:09:21.585563 29052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:21.580394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:56645 (local address 127.11.207.190:38807)
0812 21:09:21.580563 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:21.580565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:21.580575 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:09:21.580600 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:21.580601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:21.580612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:21.580714 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:21.581094 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.581646 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.582442 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.582667 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.582766 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:21.585491 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:56645: BlockingRecv error: recv got EOF from 127.11.207.129:56645 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
Aug 12 21:09:21 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:21 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:21.695583 29055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:21.690942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:35145)
0812 21:09:21.691094 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:21.691107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:21.691181 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:21.691317 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:21.691318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:21.691512 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:21.691561 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:21.691563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.692249 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.692250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:21.692872 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:21.692874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.693216 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.693217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:21.693272 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:21.693751 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:21.693756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:21.695492 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
W20260812 21:09:21.695659 29052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:21.691034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:35145 (local address 127.11.207.190:38807)
0812 21:09:21.691095 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:21.691097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:21.691160 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:09:21.691238 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:21.691239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:21.691251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:21.691299 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:21.691641 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.692163 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.692956 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.693149 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.693212 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:21.695604 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:35145: BlockingRecv error: recv got EOF from 127.11.207.130:35145 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 21:09:21 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:21 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.131@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:21.828967 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:21.824259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.131:34783)
0812 21:09:21.824405 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:21.824418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:21.824486 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:21.824622 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:21.824623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:21.824857 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:21.824920 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:21.824922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.825602 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.825603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:21.826149 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:21.826151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.826497 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.826498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:21.826555 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:21.827052 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:21.827057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:21.828886 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
W20260812 21:09:21.829046 29052 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:21.824378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:34783 (local address 127.11.207.190:38807)
0812 21:09:21.824430 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:21.824432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:21.824466 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:09:21.824548 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:21.824549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:21.824565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:21.824611 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:21.825002 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.825528 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.826228 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:21.826433 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:21.826517 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:21.828991 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.131:34783: BlockingRecv error: recv got EOF from 127.11.207.131:34783 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 21:09:22 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:22 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.129@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:22.591080 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:22.586088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.129:49599)
0812 21:09:22.586295 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:22.586306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:22.586378 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:22.586560 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:22.586561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:22.586766 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:22.586822 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:22.586825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.587595 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.587596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:22.588195 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:22.588198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.588577 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.588578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:22.588635 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:22.589082 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:22.589087 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:22.591019 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
W20260812 21:09:22.591224 29059 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:22.586190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:49599 (local address 127.11.207.190:38807)
0812 21:09:22.586318 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:22.586320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:22.586359 (+ 39us) server_negotiation.cc:407] Connection header received
0812 21:09:22.586453 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:22.586454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:22.586465 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:22.586527 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:22.586898 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.587490 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.588270 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.588486 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.588533 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:22.591152 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.129:49599: BlockingRecv error: recv got EOF from 127.11.207.129:49599 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
Aug 12 21:09:22 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:09:22 dist-test-slave-lkx2 krb5kdc[27034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.207.130@KRBTEST.COM for oryx/127.11.207.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:09:22.700927 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:22.696184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38807 (local address 127.11.207.130:40965)
0812 21:09:22.696363 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:22.696378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:22.696469 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:22.696601 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:22.696602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:22.696795 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:22.696847 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:22.696850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.697509 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.697510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:22.698061 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:22.698062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.698408 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.698410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:22.698466 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:22.698914 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:22.698920 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:22.700848 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.207.190:38807: Server oryx/127.11.207.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
W20260812 21:09:22.701004 29059 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:22.696299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:40965 (local address 127.11.207.190:38807)
0812 21:09:22.696385 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:22.696387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:22.696446 (+ 59us) server_negotiation.cc:407] Connection header received
0812 21:09:22.696527 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:22.696529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:22.696540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:22.696584 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:22.696921 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.697429 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.698144 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.698345 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.698400 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:22.700951 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.207.130:40965: BlockingRecv error: recv got EOF from 127.11.207.130:40965 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 21:09:22.742009 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 28132
I20260812 21:09:22.758466 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 28420
I20260812 21:09:22.774664 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 28671
I20260812 21:09:22.789788 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 27985
2026-08-12T21:09:22Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17682 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T21:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:22Z Disabled control of system clock
I20260812 21:09:22.831543 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:33021
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:33021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 21:09:22.913578 29071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:22.913784 29071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:22.913805 29071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:22.915032 29071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:22.915079 29071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:22.915091 29071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:22.915102 29071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:22.915112 29071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:22.916615 29071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:33021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:33021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:22.916873 29071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:22.917171 29071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:22.922300 29080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:22.922417 29081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:22.922420 29084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:22.922392 29071 server_base.cc:1034] running on GCE node
I20260812 21:09:22.922773 29071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:22.923091 29071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:22.924230 29071 hybrid_clock.cc:648] HybridClock initialized: now 1786568962924214 us; error 35 us; skew 500 ppm
I20260812 21:09:22.925446 29071 webserver.cc:460] Webserver started at http://127.11.207.190:45291/ using document root <none> and password file <none>
I20260812 21:09:22.925683 29071 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:22.925733 29071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:22.925884 29071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:22.926964 29071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "ccce089082a340ba977893d061f5d7bd"
format_stamp: "Formatted at 2026-08-12 21:09:22 on dist-test-slave-lkx2"
server_key: "e4b20084727c2954cbb0d2155a40f0bd"
server_key_iv: "d9df8500c75168afc1082e04a6d2f536"
server_key_version: "encryptionkey@0"
I20260812 21:09:22.927291 29071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "ccce089082a340ba977893d061f5d7bd"
format_stamp: "Formatted at 2026-08-12 21:09:22 on dist-test-slave-lkx2"
server_key: "e4b20084727c2954cbb0d2155a40f0bd"
server_key_iv: "d9df8500c75168afc1082e04a6d2f536"
server_key_version: "encryptionkey@0"
I20260812 21:09:22.928593 29071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:22.929358 29095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:22.929526 29071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:22.929625 29071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "ccce089082a340ba977893d061f5d7bd"
format_stamp: "Formatted at 2026-08-12 21:09:22 on dist-test-slave-lkx2"
server_key: "e4b20084727c2954cbb0d2155a40f0bd"
server_key_iv: "d9df8500c75168afc1082e04a6d2f536"
server_key_version: "encryptionkey@0"
I20260812 21:09:22.929689 29071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:22.944022 29071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:22.944372 29071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:22.944510 29071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:22.948544 29071 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:33021
I20260812 21:09:22.948588 29187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:33021 every 8 connection(s)
I20260812 21:09:22.948993 29071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:09:22.949692 29188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:22.952404 29188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd: Bootstrap starting.
I20260812 21:09:22.953019 29188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:22.953298 29188 log.cc:824] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:22.954048 29188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd: No bootstrap required, opened a new log
I20260812 21:09:22.955619 29188 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccce089082a340ba977893d061f5d7bd" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33021 } }
I20260812 21:09:22.955760 29188 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:22.955794 29188 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccce089082a340ba977893d061f5d7bd, State: Initialized, Role: FOLLOWER
I20260812 21:09:22.955948 29188 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccce089082a340ba977893d061f5d7bd" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33021 } }
I20260812 21:09:22.956040 29188 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:09:22.956075 29188 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:09:22.956125 29188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:22.956409 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 29071
I20260812 21:09:22.956490 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:09:22.956657 12094 external_mini_cluster.cc:1428] Setting key ce982aae5856037ee19af83f706ada97
I20260812 21:09:22.956791 29188 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccce089082a340ba977893d061f5d7bd" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33021 } }
I20260812 21:09:22.956914 29188 leader_election.cc:302] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccce089082a340ba977893d061f5d7bd; no voters:
I20260812 21:09:22.957079 29188 leader_election.cc:288] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:09:22.957386 29192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:22.957461 29188 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:09:22.957544 29192 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [term 1 LEADER]: Becoming Leader. State: Replica: ccce089082a340ba977893d061f5d7bd, State: Running, Role: LEADER
I20260812 21:09:22.957639 29192 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccce089082a340ba977893d061f5d7bd" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33021 } }
I20260812 21:09:22.957985 29192 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccce089082a340ba977893d061f5d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccce089082a340ba977893d061f5d7bd" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33021 } } }
I20260812 21:09:22.958061 29192 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:22.958187 29192 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccce089082a340ba977893d061f5d7bd. Latest consensus state: current_term: 1 leader_uuid: "ccce089082a340ba977893d061f5d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccce089082a340ba977893d061f5d7bd" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 33021 } } }
I20260812 21:09:22.958242 29192 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:22.958838 29207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:22.959323 29207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:22.960876 29207 catalog_manager.cc:1347] Generated new cluster ID: 0d0fe2c92344484b876688cb30ee0147
I20260812 21:09:22.960930 29207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:09:22.962522 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:22.957159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59644 (local address 127.11.207.190:33021)
0812 21:09:22.957418 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:22.957427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:22.957460 (+ 33us) server_negotiation.cc:407] Connection header received
0812 21:09:22.957581 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:22.957589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:22.957660 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:22.957803 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:22.958152 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.959595 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.960597 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:22.961081 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:22.961206 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:22.961844 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:22.961848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:22.961848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:22.962010 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:22.962042 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:22.962052 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:22.962107 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:22.962184 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 21:09:22.962355 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":125,"threads_started":1}
I20260812 21:09:22.973539 29207 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:09:22.974282 29207 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:22.980330 29207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccce089082a340ba977893d061f5d7bd: Generated new TSK 0
I20260812 21:09:22.980561 29207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:09:22.984504 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33021
--builtin_ntp_servers=127.11.207.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 21:09:23.093986 29221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.094208 29221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.094230 29221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.095435 29221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.095480 29221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.095506 29221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:09:23.096906 29221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.097154 29221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.097460 29221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.103608 29230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.103600 29233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.103600 29221 server_base.cc:1034] running on GCE node
W20260812 21:09:23.103643 29235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.104239 29221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.104606 29221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:23.105808 29221 hybrid_clock.cc:648] HybridClock initialized: now 1786568963105790 us; error 36 us; skew 500 ppm
I20260812 21:09:23.107555 29221 webserver.cc:460] Webserver started at http://127.11.207.129:34139/ using document root <none> and password file <none>
I20260812 21:09:23.107836 29221 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:23.107910 29221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:23.108053 29221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:23.109539 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "fb40a3eae37a4c3c988d2c57fee81c81"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "8392126c40d6a33ce4feeab28b467d88"
server_key_iv: "d8553e94308dddac5b0e8f75a304eed4"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.109977 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "fb40a3eae37a4c3c988d2c57fee81c81"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "8392126c40d6a33ce4feeab28b467d88"
server_key_iv: "d8553e94308dddac5b0e8f75a304eed4"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.111680 29221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:23.112623 29248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.112811 29221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:23.112907 29221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "fb40a3eae37a4c3c988d2c57fee81c81"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "8392126c40d6a33ce4feeab28b467d88"
server_key_iv: "d8553e94308dddac5b0e8f75a304eed4"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.112970 29221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:23.129427 29221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:23.129752 29221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:23.129885 29221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:23.130134 29221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:23.130498 29221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:23.130537 29221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.130558 29221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:23.130569 29221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.136530 29221 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:43851
I20260812 21:09:23.136603 29434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:43851 every 8 connection(s)
I20260812 21:09:23.136921 29221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:09:23.141639 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 29221
I20260812 21:09:23.141736 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:09:23.142025 12094 external_mini_cluster.cc:1428] Setting key a9b838466afc8916ced4c098a16c57a2
I20260812 21:09:23.142485 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.137642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:47157 (local address 127.11.207.190:33021)
0812 21:09:23.137726 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.137728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.137987 (+ 259us) server_negotiation.cc:407] Connection header received
0812 21:09:23.138517 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.138519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.138533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.138586 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.139255 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.139780 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.140933 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.141141 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.141199 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:23.142198 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:23.142200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:23.142200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:23.142206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:23.142224 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:23.142229 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:23.142271 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:23.142360 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 21:09:23.142404 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:09:23.142711 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.137626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:33021 (local address 127.11.207.129:47157)
0812 21:09:23.137868 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:23.137904 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:23.138437 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:23.138647 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:23.138650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:23.138658 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:23.139171 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:23.139176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.139897 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.139898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:23.140831 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:23.140834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.141235 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.141236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:23.141367 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:23.142024 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:09:23.142024 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:23.142056 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:09:23.142082 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:23.142281 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:23.142287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:23.142291 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:23.142375 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:09:23.142491 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260812 21:09:23.143172 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33021
--builtin_ntp_servers=127.11.207.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 21:09:23.143213 29435 heartbeater.cc:347] Connected to a master server at 127.11.207.190:33021
I20260812 21:09:23.143311 29435 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:23.143507 29435 heartbeater.cc:511] Master 127.11.207.190:33021 requested a full tablet report, sending...
I20260812 21:09:23.144085 29122 ts_manager.cc:194] Registered new tserver with Master: fb40a3eae37a4c3c988d2c57fee81c81 (127.11.207.129:43851)
I20260812 21:09:23.144989 29122 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.129:47157
W20260812 21:09:23.227761 29442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.227981 29442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.228003 29442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.229211 29442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.229256 29442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.229281 29442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:09:23.230659 29442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.230892 29442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.231184 29442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.236485 29452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.236495 29454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.236500 29450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.236513 29442 server_base.cc:1034] running on GCE node
I20260812 21:09:23.236871 29442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.237107 29442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:23.238248 29442 hybrid_clock.cc:648] HybridClock initialized: now 1786568963238216 us; error 52 us; skew 500 ppm
I20260812 21:09:23.239534 29442 webserver.cc:460] Webserver started at http://127.11.207.130:33481/ using document root <none> and password file <none>
I20260812 21:09:23.239765 29442 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:23.239814 29442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:23.239943 29442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:23.241074 29442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "4c51cee461bf4620b208d068bd267582"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "19e0eb6d0337459195a22840b539fa54"
server_key_iv: "0ba5895b550e536942890d1472961a2e"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.241432 29442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "4c51cee461bf4620b208d068bd267582"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "19e0eb6d0337459195a22840b539fa54"
server_key_iv: "0ba5895b550e536942890d1472961a2e"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.242785 29442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:09:23.243551 29464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.243723 29442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:09:23.243813 29442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "4c51cee461bf4620b208d068bd267582"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "19e0eb6d0337459195a22840b539fa54"
server_key_iv: "0ba5895b550e536942890d1472961a2e"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.243888 29442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:23.255946 29442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:23.256287 29442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:23.256464 29442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:23.256701 29442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:23.257030 29442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:23.257064 29442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.257093 29442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:23.257117 29442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.263090 29442 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:32931
I20260812 21:09:23.263132 29645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:32931 every 8 connection(s)
I20260812 21:09:23.263479 29442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:09:23.268560 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.264269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:60915 (local address 127.11.207.190:33021)
0812 21:09:23.264328 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.264331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.264587 (+ 256us) server_negotiation.cc:407] Connection header received
0812 21:09:23.265093 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.265094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.265106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.265172 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.265820 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.266339 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.267276 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.267476 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.267543 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:23.268275 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:23.268276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:23.268276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:23.268281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:23.268299 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:23.268303 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:23.268404 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:23.268466 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 21:09:23.268493 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:09:23.268747 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.264251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:33021 (local address 127.11.207.130:60915)
0812 21:09:23.264463 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:23.264504 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:23.265000 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:23.265225 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:23.265228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:23.265238 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:23.265723 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:23.265728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.266415 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.266415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:23.267192 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:23.267194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.267550 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.267551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:23.267658 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:23.268120 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:09:23.268120 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:23.268149 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:09:23.268172 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:23.268405 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:23.268408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:23.268410 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:09:23.268458 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:09:23.268540 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260812 21:09:23.269275 29646 heartbeater.cc:347] Connected to a master server at 127.11.207.190:33021
I20260812 21:09:23.269369 29646 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:23.269551 29646 heartbeater.cc:511] Master 127.11.207.190:33021 requested a full tablet report, sending...
I20260812 21:09:23.269680 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 29442
I20260812 21:09:23.269773 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:09:23.269989 12094 external_mini_cluster.cc:1428] Setting key 33cac147291d6fbbbf88026a9f13d07e
I20260812 21:09:23.269972 29122 ts_manager.cc:194] Registered new tserver with Master: 4c51cee461bf4620b208d068bd267582 (127.11.207.130:32931)
I20260812 21:09:23.270498 29122 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.130:60915
I20260812 21:09:23.271006 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33021
--builtin_ntp_servers=127.11.207.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 21:09:23.355199 29651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.355425 29651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.355446 29651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.356662 29651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.356703 29651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.356734 29651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:09:23.358239 29651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:33021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.358496 29651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.358803 29651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.363907 29662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.363901 29660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.363989 29651 server_base.cc:1034] running on GCE node
W20260812 21:09:23.363901 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.364359 29651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.364598 29651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:23.365772 29651 hybrid_clock.cc:648] HybridClock initialized: now 1786568963365756 us; error 27 us; skew 500 ppm
I20260812 21:09:23.367175 29651 webserver.cc:460] Webserver started at http://127.11.207.131:45081/ using document root <none> and password file <none>
I20260812 21:09:23.367408 29651 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:23.367455 29651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:23.367573 29651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:23.368769 29651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "5f3e372f550540edb6c276a23a02f419"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "b99769702478b11f3e045c27f99d9dcc"
server_key_iv: "3f5b3297b083cf6d885a1756f41f997f"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.369118 29651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "5f3e372f550540edb6c276a23a02f419"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "b99769702478b11f3e045c27f99d9dcc"
server_key_iv: "3f5b3297b083cf6d885a1756f41f997f"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.370438 29651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:09:23.371205 29675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.371362 29651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:23.371452 29651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "5f3e372f550540edb6c276a23a02f419"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "b99769702478b11f3e045c27f99d9dcc"
server_key_iv: "3f5b3297b083cf6d885a1756f41f997f"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.371517 29651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:23.393581 29651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:23.393922 29651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:23.394060 29651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:23.394301 29651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:23.394649 29651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:23.394742 29651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.394780 29651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:23.394804 29651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.400364 29651 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:42991
I20260812 21:09:23.400424 29892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:42991 every 8 connection(s)
I20260812 21:09:23.400789 29651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:09:23.405954 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.401577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:36567 (local address 127.11.207.190:33021)
0812 21:09:23.401663 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.401665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.401845 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:09:23.402336 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.402337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.402348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.402402 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.403075 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.403588 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.404627 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.404861 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.404918 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:23.405745 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:23.405746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:23.405746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:23.405751 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:23.405763 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:23.405766 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:23.405798 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:23.405868 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 21:09:23.405893 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 21:09:23.406160 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.401499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:33021 (local address 127.11.207.131:36567)
0812 21:09:23.401705 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:23.401751 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:23.402259 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:23.402458 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:23.402461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:23.402470 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:23.402989 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:23.402994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.403676 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.403677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:23.404550 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:23.404552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:23.404933 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:23.404934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:23.405052 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:23.405612 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:09:23.405612 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:23.405643 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:09:23.405667 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:23.405810 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:23.405814 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:23.405817 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:09:23.405866 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:09:23.405951 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260812 21:09:23.406662 29893 heartbeater.cc:347] Connected to a master server at 127.11.207.190:33021
I20260812 21:09:23.406759 29893 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:23.406751 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 29651
I20260812 21:09:23.406831 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:09:23.406956 29893 heartbeater.cc:511] Master 127.11.207.190:33021 requested a full tablet report, sending...
I20260812 21:09:23.407019 12094 external_mini_cluster.cc:1428] Setting key 93bd435a0e529b35142e760dd3b7b7e6
I20260812 21:09:23.407408 29122 ts_manager.cc:194] Registered new tserver with Master: 5f3e372f550540edb6c276a23a02f419 (127.11.207.131:42991)
I20260812 21:09:23.407863 29122 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.131:36567
I20260812 21:09:23.407883 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 21:09:23.409649 29194 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:23.409237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.11.207.190:33021)
0812 21:09:23.409279 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.409281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.409387 (+ 106us) server_negotiation.cc:407] Connection header received
0812 21:09:23.409422 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.409423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.409432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.409479 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.409592 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59648: BlockingRecv error: recv got EOF from 127.0.0.1:59648 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:09:23.409742 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 29221
I20260812 21:09:23.428578 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 29442
I20260812 21:09:23.445422 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 29651
I20260812 21:09:23.458992 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 29071
2026-08-12T21:09:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (670 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T21:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:23Z Disabled control of system clock
I20260812 21:09:23.501494 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:39029
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:39029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:09:23.583324 29907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.583530 29907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.583551 29907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.584704 29907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:23.584748 29907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.584759 29907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.584770 29907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:23.584782 29907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:23.586143 29907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:39029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.586345 29907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.586596 29907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.591547 29919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.591519 29925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.591657 29907 server_base.cc:1034] running on GCE node
W20260812 21:09:23.591540 29922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.591995 29907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.592290 29907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:23.593459 29907 hybrid_clock.cc:648] HybridClock initialized: now 1786568963593442 us; error 35 us; skew 500 ppm
I20260812 21:09:23.594681 29907 webserver.cc:460] Webserver started at http://127.11.207.190:34749/ using document root <none> and password file <none>
I20260812 21:09:23.594921 29907 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:23.594969 29907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:23.595081 29907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:23.596153 29907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "c5f8aefdc7524f18b3ec8a1895bdb397"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "186376380aa36ad604415236019b88ad"
server_key_iv: "2801fde1f6b92421101bfbc5050f0564"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.596527 29907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "c5f8aefdc7524f18b3ec8a1895bdb397"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "186376380aa36ad604415236019b88ad"
server_key_iv: "2801fde1f6b92421101bfbc5050f0564"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.597775 29907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:23.598512 29935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.598692 29907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:23.598783 29907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "c5f8aefdc7524f18b3ec8a1895bdb397"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "186376380aa36ad604415236019b88ad"
server_key_iv: "2801fde1f6b92421101bfbc5050f0564"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.598846 29907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:23.602186 29907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:23.602454 29907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:23.602571 29907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:23.606416 29907 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:39029
I20260812 21:09:23.606447 30038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:39029 every 8 connection(s)
I20260812 21:09:23.606755 29907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:09:23.607384 30039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:23.610198 30039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397: Bootstrap starting.
I20260812 21:09:23.610826 30039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:23.611215 30039 log.cc:824] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:23.611891 30039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397: No bootstrap required, opened a new log
I20260812 21:09:23.613580 30039 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39029 } }
I20260812 21:09:23.613724 30039 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:23.613747 30039 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f8aefdc7524f18b3ec8a1895bdb397, State: Initialized, Role: FOLLOWER
I20260812 21:09:23.613895 30039 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39029 } }
I20260812 21:09:23.613961 30039 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:09:23.613983 30039 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:09:23.614030 30039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:23.614584 30039 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39029 } }
I20260812 21:09:23.614677 30039 leader_election.cc:302] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5f8aefdc7524f18b3ec8a1895bdb397; no voters:
I20260812 21:09:23.614823 30039 leader_election.cc:288] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:09:23.614874 30046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:23.615034 30046 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [term 1 LEADER]: Becoming Leader. State: Replica: c5f8aefdc7524f18b3ec8a1895bdb397, State: Running, Role: LEADER
I20260812 21:09:23.615089 30039 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:09:23.615129 30046 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39029 } }
I20260812 21:09:23.615486 30047 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39029 } } }
I20260812 21:09:23.615649 30047 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:23.615760 30048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5f8aefdc7524f18b3ec8a1895bdb397. Latest consensus state: current_term: 1 leader_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f8aefdc7524f18b3ec8a1895bdb397" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39029 } } }
I20260812 21:09:23.615842 30048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397 [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:23.616118 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 29907
I20260812 21:09:23.616199 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:09:23.616392 12094 external_mini_cluster.cc:1428] Setting key 32495c12208940fc2e6b781c2bb1a287
I20260812 21:09:23.616750 30060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:23.617367 30060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:23.619165 30060 catalog_manager.cc:1347] Generated new cluster ID: ff04a0036e444c52b70987088f530c08
I20260812 21:09:23.619221 30060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:09:23.626142 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.617031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.11.207.190:39029)
0812 21:09:23.624519 (+ 7488us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.624529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.624573 (+ 44us) server_negotiation.cc:407] Connection header received
0812 21:09:23.624712 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.624719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.624766 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.624925 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.625672 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:23.625677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:23.625678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:23.625682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:23.625723 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:23.625737 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:23.625810 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:23.625870 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 21:09:23.625932 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7395,"thread_start_us":108,"threads_started":1}
I20260812 21:09:23.660084 30060 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:09:23.660290 30060 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:23.666647 30060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5f8aefdc7524f18b3ec8a1895bdb397: Generated new TSK 0
I20260812 21:09:23.666886 30060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:09:23.674851 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:39029
--builtin_ntp_servers=127.11.207.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:09:23.758809 30090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.759029 30090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.759052 30090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.760234 30090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.760277 30090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.760303 30090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:09:23.761708 30090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.761942 30090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.762233 30090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.767248 30098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.767222 30096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.767239 30102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.767518 30090 server_base.cc:1034] running on GCE node
I20260812 21:09:23.767746 30090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.767982 30090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:23.769142 30090 hybrid_clock.cc:648] HybridClock initialized: now 1786568963769128 us; error 25 us; skew 500 ppm
I20260812 21:09:23.770488 30090 webserver.cc:460] Webserver started at http://127.11.207.129:39403/ using document root <none> and password file <none>
I20260812 21:09:23.770710 30090 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:23.770758 30090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:23.770890 30090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:23.771956 30090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "e8f816d3eb6a4280bab4c1d3d4724fdb"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "d263b62b59df42c84e0321da3de59743"
server_key_iv: "53c07944057e539771af477216458e29"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.772303 30090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "e8f816d3eb6a4280bab4c1d3d4724fdb"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "d263b62b59df42c84e0321da3de59743"
server_key_iv: "53c07944057e539771af477216458e29"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.773584 30090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:23.774281 30112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.774430 30090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:09:23.774516 30090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "e8f816d3eb6a4280bab4c1d3d4724fdb"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "d263b62b59df42c84e0321da3de59743"
server_key_iv: "53c07944057e539771af477216458e29"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.774586 30090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:23.777945 30090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:23.778198 30090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:23.778317 30090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:23.778517 30090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:23.778823 30090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:23.778862 30090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.778889 30090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:23.778913 30090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.785977 30090 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:37903
I20260812 21:09:23.786024 30323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:37903 every 8 connection(s)
I20260812 21:09:23.786446 30090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:09:23.789568 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.787187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:50937 (local address 127.11.207.190:39029)
0812 21:09:23.787246 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.787248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.787465 (+ 217us) server_negotiation.cc:407] Connection header received
0812 21:09:23.788043 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.788044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.788053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.788106 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.789322 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:23.789323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:23.789324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:23.789329 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:23.789348 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:23.789352 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:23.789406 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:23.789477 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:09:23.789494 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:09:23.789772 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.787156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:39029 (local address 127.11.207.129:50937)
0812 21:09:23.787354 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:23.787388 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:23.787942 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:23.788179 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:23.788183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:23.788193 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:23.789151 (+ 958us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:09:23.789152 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:23.789194 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:09:23.789232 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:23.789409 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:23.789416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:23.789420 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:23.789490 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:09:23.789553 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260812 21:09:23.790215 30327 heartbeater.cc:347] Connected to a master server at 127.11.207.190:39029
I20260812 21:09:23.790306 30327 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:23.790459 30327 heartbeater.cc:511] Master 127.11.207.190:39029 requested a full tablet report, sending...
I20260812 21:09:23.790772 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 30090
I20260812 21:09:23.790853 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:09:23.791033 12094 external_mini_cluster.cc:1428] Setting key f8499c0173f568e264290bf017cfbd69
I20260812 21:09:23.791019 29965 ts_manager.cc:194] Registered new tserver with Master: e8f816d3eb6a4280bab4c1d3d4724fdb (127.11.207.129:37903)
I20260812 21:09:23.791936 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:39029
--builtin_ntp_servers=127.11.207.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:09:23.876197 30336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.876439 30336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.876463 30336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.877769 30336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.877818 30336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.877846 30336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:09:23.879298 30336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.879550 30336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.879863 30336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.884989 30345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.884989 30347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.885149 30336 server_base.cc:1034] running on GCE node
W20260812 21:09:23.884979 30350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.885375 30336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.885609 30336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:23.886759 30336 hybrid_clock.cc:648] HybridClock initialized: now 1786568963886744 us; error 31 us; skew 500 ppm
I20260812 21:09:23.888043 30336 webserver.cc:460] Webserver started at http://127.11.207.130:45027/ using document root <none> and password file <none>
I20260812 21:09:23.888284 30336 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:23.888336 30336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:23.888497 30336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:23.889644 30336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "8f1ce4e2dd494488ab17db44901502f1"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "d655598e56b5ae5a2327647863e2d4ba"
server_key_iv: "61f2eea412c78fa6821764d50610fbab"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.889991 30336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "8f1ce4e2dd494488ab17db44901502f1"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "d655598e56b5ae5a2327647863e2d4ba"
server_key_iv: "61f2eea412c78fa6821764d50610fbab"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.891360 30336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:23.892083 30362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.892261 30336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:23.892391 30336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "8f1ce4e2dd494488ab17db44901502f1"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "d655598e56b5ae5a2327647863e2d4ba"
server_key_iv: "61f2eea412c78fa6821764d50610fbab"
server_key_version: "encryptionkey@0"
I20260812 21:09:23.892457 30336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:23.895783 30336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:23.896056 30336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:23.896180 30336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:23.896401 30336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:23.896715 30336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:23.896750 30336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.896778 30336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:23.896801 30336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:23.902503 30336 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:38605
I20260812 21:09:23.902562 30567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:38605 every 8 connection(s)
I20260812 21:09:23.902904 30336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:09:23.905797 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.903590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:46013 (local address 127.11.207.190:39029)
0812 21:09:23.903706 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:23.903708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:23.903924 (+ 216us) server_negotiation.cc:407] Connection header received
0812 21:09:23.904405 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:23.904406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:23.904413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:23.904472 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:23.905560 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:23.905561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:23.905561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:23.905566 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:23.905584 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:23.905589 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:23.905648 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:23.905723 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 21:09:23.905735 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 21:09:23.905961 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:23.903580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:39029 (local address 127.11.207.130:46013)
0812 21:09:23.903776 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:23.903820 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:23.904293 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:23.904539 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:23.904543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:23.904551 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:23.905411 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:09:23.905412 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:23.905440 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:09:23.905469 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:23.905649 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:23.905651 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:23.905653 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:09:23.905705 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:09:23.905754 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260812 21:09:23.906440 30568 heartbeater.cc:347] Connected to a master server at 127.11.207.190:39029
I20260812 21:09:23.906519 30568 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:23.906637 30568 heartbeater.cc:511] Master 127.11.207.190:39029 requested a full tablet report, sending...
I20260812 21:09:23.906939 29965 ts_manager.cc:194] Registered new tserver with Master: 8f1ce4e2dd494488ab17db44901502f1 (127.11.207.130:38605)
I20260812 21:09:23.907433 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 30336
I20260812 21:09:23.907519 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:09:23.907719 12094 external_mini_cluster.cc:1428] Setting key fc7f73a47c9f8470090d4e5249c8fe90
I20260812 21:09:23.908583 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:39029
--builtin_ntp_servers=127.11.207.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:09:23.990146 30572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:23.990350 30572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:23.990373 30572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:23.991493 30572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:23.991524 30572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:23.991547 30572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:09:23.992913 30572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:23.993114 30572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:23.993371 30572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:23.998276 30595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.998289 30582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:23.998356 30587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:23.998377 30572 server_base.cc:1034] running on GCE node
I20260812 21:09:23.998703 30572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:23.998935 30572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:24.000080 30572 hybrid_clock.cc:648] HybridClock initialized: now 1786568964000059 us; error 33 us; skew 500 ppm
I20260812 21:09:24.001710 30572 webserver.cc:460] Webserver started at http://127.11.207.131:44419/ using document root <none> and password file <none>
I20260812 21:09:24.002007 30572 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:24.002079 30572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:24.002228 30572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:24.003352 30572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "61f391e8255e4f34bd5b6bc78a5568d2"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "4e1e72409046331cac8e6c2226a8f39c"
server_key_iv: "9c24ab94466e529b39f52d3e9689f56c"
server_key_version: "encryptionkey@0"
I20260812 21:09:24.003682 30572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "61f391e8255e4f34bd5b6bc78a5568d2"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "4e1e72409046331cac8e6c2226a8f39c"
server_key_iv: "9c24ab94466e529b39f52d3e9689f56c"
server_key_version: "encryptionkey@0"
I20260812 21:09:24.004926 30572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 21:09:24.005553 30606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:24.005685 30572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:24.005770 30572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "61f391e8255e4f34bd5b6bc78a5568d2"
format_stamp: "Formatted at 2026-08-12 21:09:23 on dist-test-slave-lkx2"
server_key: "4e1e72409046331cac8e6c2226a8f39c"
server_key_iv: "9c24ab94466e529b39f52d3e9689f56c"
server_key_version: "encryptionkey@0"
I20260812 21:09:24.005841 30572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:24.009135 30572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:24.009382 30572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:24.009491 30572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:24.009673 30572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:24.009956 30572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:24.009987 30572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:24.010016 30572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:24.010040 30572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:24.015620 30572 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:44079
I20260812 21:09:24.015703 30821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:44079 every 8 connection(s)
I20260812 21:09:24.016047 30572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:09:24.019069 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:24.016797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:59835 (local address 127.11.207.190:39029)
0812 21:09:24.016853 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:24.016855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:24.017113 (+ 258us) server_negotiation.cc:407] Connection header received
0812 21:09:24.017615 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:24.017616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:24.017625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:24.017744 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:24.018830 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:24.018832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:24.018832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:09:24.018837 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:24.018854 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:09:24.018859 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:24.018910 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:24.018983 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 21:09:24.018998 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:09:24.019281 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:24.016820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:39029 (local address 127.11.207.131:59835)
0812 21:09:24.017000 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:24.017034 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:24.017517 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:24.017836 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:24.017840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:24.017848 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:24.018684 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:09:24.018684 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:24.018713 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:09:24.018742 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:24.018915 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:24.018919 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:24.018923 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:24.018989 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:09:24.019055 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260812 21:09:24.019718 30825 heartbeater.cc:347] Connected to a master server at 127.11.207.190:39029
I20260812 21:09:24.019804 30825 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:24.019917 30825 heartbeater.cc:511] Master 127.11.207.190:39029 requested a full tablet report, sending...
I20260812 21:09:24.020256 29965 ts_manager.cc:194] Registered new tserver with Master: 61f391e8255e4f34bd5b6bc78a5568d2 (127.11.207.131:44079)
I20260812 21:09:24.024051 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 30572
I20260812 21:09:24.024158 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:09:24.024302 12094 external_mini_cluster.cc:1428] Setting key 6434586aba6c193686a446080c82d9b6
I20260812 21:09:24.025208 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 21:09:24.027213 30082 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:09:24.026634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.11.207.190:39029)
0812 21:09:24.026740 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:24.026742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:24.026824 (+ 82us) server_negotiation.cc:407] Connection header received
0812 21:09:24.026893 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:24.026894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:24.026900 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:24.026944 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:24.027149 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59228: BlockingRecv error: recv got EOF from 127.0.0.1:59228 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 21:09:24.027256 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 30090
I20260812 21:09:24.036413 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 30336
I20260812 21:09:24.063441 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 30572
I20260812 21:09:24.072438 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 29907
2026-08-12T21:09:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (603 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:09:24 dist-test-slave-lkx2 krb5kdc[30840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:09:24 dist-test-slave-lkx2 krb5kdc[30840](info): set up 2 sockets
Aug 12 21:09:24 dist-test-slave-lkx2 krb5kdc[30840](info): commencing operation
krb5kdc: starting...
W20260812 21:09:25.289193 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:09:25.413540 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43447
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:25.497174 30865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:25.497390 30865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:25.497411 30865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:25.498584 30865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:09:25.498627 30865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:25.498639 30865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:25.498651 30865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:09:25.498662 30865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:09:25.500232 30865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:25.500504 30865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:25.500728 30865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:25.505597 30875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:25.505640 30878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:25.505652 30874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:25.505689 30865 server_base.cc:1034] running on GCE node
I20260812 21:09:25.505998 30865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:25.506302 30865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:25.507469 30865 hybrid_clock.cc:648] HybridClock initialized: now 1786568965507453 us; error 30 us; skew 500 ppm
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:25.510027 30865 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:09:25.510450 30865 webserver.cc:460] Webserver started at http://127.11.207.190:43491/ using document root <none> and password file <none>
I20260812 21:09:25.510655 30865 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:25.510695 30865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:25.510798 30865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:25.511853 30865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "1c7eacc949f14b4987c5086505bece6d"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "53d775b1cfa9d8e9faccee1762c2ca2f"
server_key_iv: "40c29f15ee87842c843fa5b1a791d7e0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.512136 30865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "1c7eacc949f14b4987c5086505bece6d"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "53d775b1cfa9d8e9faccee1762c2ca2f"
server_key_iv: "40c29f15ee87842c843fa5b1a791d7e0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.513442 30865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:25.514158 30891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.514308 30865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:25.514397 30865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "1c7eacc949f14b4987c5086505bece6d"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "53d775b1cfa9d8e9faccee1762c2ca2f"
server_key_iv: "40c29f15ee87842c843fa5b1a791d7e0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.514464 30865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:25.523065 30865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:25.524804 30865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:25.524926 30865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:25.528828 30865 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43447
I20260812 21:09:25.528860 30992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43447 every 8 connection(s)
I20260812 21:09:25.529186 30865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:09:25.530171 30993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:25.532449 30993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d: Bootstrap starting.
I20260812 21:09:25.532994 30993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:25.533226 30993 log.cc:824] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:25.533972 30993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d: No bootstrap required, opened a new log
I20260812 21:09:25.535560 30993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7eacc949f14b4987c5086505bece6d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43447 } }
I20260812 21:09:25.535708 30993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:25.535737 30993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c7eacc949f14b4987c5086505bece6d, State: Initialized, Role: FOLLOWER
I20260812 21:09:25.535837 30993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7eacc949f14b4987c5086505bece6d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43447 } }
I20260812 21:09:25.535888 30993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:09:25.535907 30993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:09:25.535938 30993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:25.536787 30993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7eacc949f14b4987c5086505bece6d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43447 } }
I20260812 21:09:25.536880 30993 leader_election.cc:302] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c7eacc949f14b4987c5086505bece6d; no voters:
I20260812 21:09:25.537002 30993 leader_election.cc:288] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:09:25.537072 31002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:25.537241 30993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:09:25.537263 31002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [term 1 LEADER]: Becoming Leader. State: Replica: 1c7eacc949f14b4987c5086505bece6d, State: Running, Role: LEADER
I20260812 21:09:25.537358 31002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7eacc949f14b4987c5086505bece6d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43447 } }
I20260812 21:09:25.537746 31003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c7eacc949f14b4987c5086505bece6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7eacc949f14b4987c5086505bece6d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43447 } } }
I20260812 21:09:25.537757 31005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c7eacc949f14b4987c5086505bece6d. Latest consensus state: current_term: 1 leader_uuid: "1c7eacc949f14b4987c5086505bece6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c7eacc949f14b4987c5086505bece6d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43447 } } }
I20260812 21:09:25.537856 31003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:25.537869 31005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d [sys.catalog]: This master's current role is: LEADER
I20260812 21:09:25.538100 31017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:09:25.538666 31017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:09:25.539383 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 30865
I20260812 21:09:25.539501 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:09:25.539685 12094 external_mini_cluster.cc:1428] Setting key 79fd5f9be583f2c3d0e6c43d48e8e005
I20260812 21:09:25.540510 31017 catalog_manager.cc:1347] Generated new cluster ID: 34f0b3cfcd244f5bb83088aa3d7e4932
I20260812 21:09:25.540565 31017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:25.549829 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:25.540200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50288 (local address 127.11.207.190:43447)
0812 21:09:25.540378 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:25.540388 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:25.540596 (+ 208us) server_negotiation.cc:407] Connection header received
0812 21:09:25.540735 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:25.540742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:25.540787 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:25.540915 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:25.541586 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.542534 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.543325 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.543784 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.543894 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:25.546402 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:25.546407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:25.546407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:25.546422 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:25.548111 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:25.548539 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:25.548543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:25.548544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:25.548660 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:25.548873 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:25.548874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:25.548874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:25.549147 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:25.549320 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:25.549434 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 21:09:25.549581 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260812 21:09:25.568598 31017 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:09:25.569206 31017 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:09:25.584591 31017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d: Generated new TSK 0
I20260812 21:09:25.584842 31017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:09:25.621291 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43447
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:25.707015 31043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:25.707212 31043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:25.707235 31043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:25.708421 31043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:25.708468 31043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:25.708495 31043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:09:25.710009 31043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:25.710248 31043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:25.710484 31043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:25.715631 31055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:25.715715 31043 server_base.cc:1034] running on GCE node
W20260812 21:09:25.715588 31058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:25.715622 31051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:25.716001 31043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:25.716246 31043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:25.717386 31043 hybrid_clock.cc:648] HybridClock initialized: now 1786568965717363 us; error 60 us; skew 500 ppm
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:25.723321 31043 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:09:25.723757 31043 webserver.cc:460] Webserver started at http://127.11.207.129:43811/ using document root <none> and password file <none>
I20260812 21:09:25.724015 31043 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:25.724068 31043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:25.724198 31043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:25.725293 31043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "9164bfdf6e7442f3ac757c3674932e02"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "7d38a23eed11344e5f302a5ac31c5264"
server_key_iv: "9a918924c745827a93102d90032f5ae0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.725639 31043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "9164bfdf6e7442f3ac757c3674932e02"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "7d38a23eed11344e5f302a5ac31c5264"
server_key_iv: "9a918924c745827a93102d90032f5ae0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.726938 31043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:09:25.727641 31069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.727790 31043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:25.727880 31043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "9164bfdf6e7442f3ac757c3674932e02"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "7d38a23eed11344e5f302a5ac31c5264"
server_key_iv: "9a918924c745827a93102d90032f5ae0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.727948 31043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:25.754379 31043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:25.756637 31043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:25.756786 31043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:25.757025 31043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:25.757356 31043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:25.757391 31043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.757421 31043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:25.757444 31043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.763134 31043 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:45821
I20260812 21:09:25.763195 31276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:45821 every 8 connection(s)
I20260812 21:09:25.763512 31043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:09:25.767946 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 31043
I20260812 21:09:25.768038 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:09:25.768200 12094 external_mini_cluster.cc:1428] Setting key 57128814c73b1e64751a0070e936784e
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:25.773008 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:25.764235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:40159 (local address 127.11.207.190:43447)
0812 21:09:25.764309 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:25.764311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:25.764565 (+ 254us) server_negotiation.cc:407] Connection header received
0812 21:09:25.765029 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:25.765031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:25.765046 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:25.765108 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:25.766093 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.766612 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.767573 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.767814 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.767879 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:25.770284 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:25.770286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:25.770287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:25.770293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:25.771571 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:25.772026 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:25.772028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:25.772028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:25.772089 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:25.772388 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:25.772389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:25.772389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:25.772495 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:25.772549 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:25.772867 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 21:09:25.772943 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:09:25.773267 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:25.764188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43447 (local address 127.11.207.129:40159)
0812 21:09:25.764402 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:25.764445 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:25.764948 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:25.765173 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:25.765177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:25.765744 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:25.765993 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:25.765999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.766676 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.766677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:25.767500 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:25.767502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.767894 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.767894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:25.768006 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:25.768584 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:25.768595 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:25.770194 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:25.771654 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:25.771662 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:25.771664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:25.771950 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:25.772152 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:25.772153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:25.772153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:25.772299 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:25.772553 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:25.772559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:25.772725 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 21:09:25.772843 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 21:09:25.773013 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260812 21:09:25.773924 31280 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43447
I20260812 21:09:25.774035 31280 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:25.774246 31280 heartbeater.cc:511] Master 127.11.207.190:43447 requested a full tablet report, sending...
I20260812 21:09:25.774789 30922 ts_manager.cc:194] Registered new tserver with Master: 9164bfdf6e7442f3ac757c3674932e02 (127.11.207.129:45821)
I20260812 21:09:25.775586 30922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:40159
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:09:25.799016 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43447
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:25.883255 31291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:25.883466 31291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:25.883489 31291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:25.884682 31291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:25.884727 31291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:25.884753 31291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:09:25.886215 31291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:25.886406 31291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:25.886590 31291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:25.891569 31307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:25.891554 31299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:25.891554 31300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:25.891587 31291 server_base.cc:1034] running on GCE node
I20260812 21:09:25.891944 31291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:25.892179 31291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:25.893332 31291 hybrid_clock.cc:648] HybridClock initialized: now 1786568965893299 us; error 46 us; skew 500 ppm
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:25.895761 31291 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:09:25.896200 31291 webserver.cc:460] Webserver started at http://127.11.207.130:33135/ using document root <none> and password file <none>
I20260812 21:09:25.896472 31291 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:25.896524 31291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:25.896656 31291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:25.897702 31291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "37c0ae5d8730d72847920fc73546d0db"
server_key_iv: "5580a5486e79aee4c4f116f7d9033fc0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.898039 31291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "37c0ae5d8730d72847920fc73546d0db"
server_key_iv: "5580a5486e79aee4c4f116f7d9033fc0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.899434 31291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:09:25.900072 31322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.900231 31291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:09:25.900316 31291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4"
format_stamp: "Formatted at 2026-08-12 21:09:25 on dist-test-slave-lkx2"
server_key: "37c0ae5d8730d72847920fc73546d0db"
server_key_iv: "5580a5486e79aee4c4f116f7d9033fc0"
server_key_version: "encryptionkey@0"
I20260812 21:09:25.900413 31291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:25.920107 31291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:25.921981 31291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:25.922119 31291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:25.922353 31291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:25.922660 31291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:25.922694 31291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.922724 31291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:25.922748 31291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:25.928802 31291 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:36933
I20260812 21:09:25.928858 31524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:36933 every 8 connection(s)
I20260812 21:09:25.929215 31291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:09:25.935117 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 31291
I20260812 21:09:25.935194 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
Aug 12 21:09:25 dist-test-slave-lkx2 krb5kdc[30840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568965, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:25.935355 12094 external_mini_cluster.cc:1428] Setting key 1dea8477ad1afd026db825ed1f6cfaf1
I20260812 21:09:25.938616 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:25.929898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:47391 (local address 127.11.207.190:43447)
0812 21:09:25.929960 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:25.929962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:25.930138 (+ 176us) server_negotiation.cc:407] Connection header received
0812 21:09:25.930723 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:25.930725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:25.930738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:25.930791 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:25.931771 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.932301 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.933299 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.933518 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.933577 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:25.935730 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:25.935731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:25.935731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:25.935739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:25.937119 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:25.937597 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:25.937598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:25.937598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:25.937679 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:25.937947 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:25.937948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:25.937949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:25.938096 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:25.938165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:25.938437 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 21:09:25.938527 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:09:25.938719 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:25.929848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43447 (local address 127.11.207.130:47391)
0812 21:09:25.930021 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:25.930060 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:25.930605 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:25.930865 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:25.930869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:25.931451 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:25.931680 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:25.931685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.932404 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.932405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:25.933213 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:25.933216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:25.933590 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:25.933592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:25.933714 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:25.934200 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:25.934209 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:25.935668 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:25.937215 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:25.937224 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:25.937227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:25.937526 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:25.937730 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:25.937731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:25.937731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:25.937877 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:25.938159 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:25.938164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:25.938300 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 21:09:25.938404 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 21:09:25.938533 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260812 21:09:25.939177 31525 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43447
I20260812 21:09:25.939276 31525 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:25.939462 31525 heartbeater.cc:511] Master 127.11.207.190:43447 requested a full tablet report, sending...
I20260812 21:09:25.939848 30922 ts_manager.cc:194] Registered new tserver with Master: 6fd96a5dcfd34be8bb499ae4d56c83d4 (127.11.207.130:36933)
I20260812 21:09:25.940303 30922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:47391
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:09:25.961391 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43447
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:09:26.045480 31534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:09:26.045691 31534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:09:26.045713 31534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:09:26.046855 31534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:09:26.046895 31534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:09:26.046921 31534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:09:26.048384 31534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:09:26.048619 31534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:09:26.048830 31534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:09:26.053853 31550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:26.053828 31544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:09:26.053821 31545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:09:26.054054 31534 server_base.cc:1034] running on GCE node
I20260812 21:09:26.054250 31534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:09:26.054477 31534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:09:26.055630 31534 hybrid_clock.cc:648] HybridClock initialized: now 1786568966055615 us; error 30 us; skew 500 ppm
Aug 12 21:09:26 dist-test-slave-lkx2 krb5kdc[30840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:09:26.058364 31534 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:09:26.058759 31534 webserver.cc:460] Webserver started at http://127.11.207.131:45539/ using document root <none> and password file <none>
I20260812 21:09:26.058939 31534 fs_manager.cc:359] Metadata directory not provided
I20260812 21:09:26.058979 31534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:09:26.059068 31534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:09:26.060079 31534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "f646c8217d07490dafde4373d676eb40"
format_stamp: "Formatted at 2026-08-12 21:09:26 on dist-test-slave-lkx2"
server_key: "dc47199ca347ffa2c1e6165892685614"
server_key_iv: "171b4e73c67f2b8e792ca712c61403f8"
server_key_version: "encryptionkey@0"
I20260812 21:09:26.060500 31534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "f646c8217d07490dafde4373d676eb40"
format_stamp: "Formatted at 2026-08-12 21:09:26 on dist-test-slave-lkx2"
server_key: "dc47199ca347ffa2c1e6165892685614"
server_key_iv: "171b4e73c67f2b8e792ca712c61403f8"
server_key_version: "encryptionkey@0"
I20260812 21:09:26.061738 31534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:09:26.062383 31563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:26.062543 31534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:09:26.062629 31534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "f646c8217d07490dafde4373d676eb40"
format_stamp: "Formatted at 2026-08-12 21:09:26 on dist-test-slave-lkx2"
server_key: "dc47199ca347ffa2c1e6165892685614"
server_key_iv: "171b4e73c67f2b8e792ca712c61403f8"
server_key_version: "encryptionkey@0"
I20260812 21:09:26.062686 31534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:09:26.110500 31534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:09:26.112790 31534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:09:26.112916 31534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:09:26.113140 31534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:09:26.113490 31534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:09:26.113526 31534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:26.113549 31534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:09:26.113574 31534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:09:26.119397 31534 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:44343
I20260812 21:09:26.119454 31781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:44343 every 8 connection(s)
I20260812 21:09:26.119755 31534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:09:26 dist-test-slave-lkx2 krb5kdc[30840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568966, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:09:26.127121 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 31534
I20260812 21:09:26.127218 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:09:26.127390 12094 external_mini_cluster.cc:1428] Setting key f66d33b6896dd588ebcc3c72b8427c3e
I20260812 21:09:26.128768 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.120466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:35629 (local address 127.11.207.190:43447)
0812 21:09:26.120527 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.120529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.120740 (+ 211us) server_negotiation.cc:407] Connection header received
0812 21:09:26.121225 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.121227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.121238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.121289 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:26.122203 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.122835 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.123831 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.124021 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.124083 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.126129 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:26.126131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:26.126131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:26.126138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:26.127400 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:26.127780 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:26.127781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:26.127781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:26.127846 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:26.128120 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:26.128121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:26.128122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:26.128260 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:26.128328 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.128631 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.128704 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:09:26.128948 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.120456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43447 (local address 127.11.207.131:35629)
0812 21:09:26.120616 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.120655 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:26.121148 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:26.121350 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:26.121353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:26.121933 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:09:26.122136 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.122141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.122930 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.122931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.123723 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.123726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.124089 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.124089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.124188 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.124644 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:09:26.124653 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:09:26.126041 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:09:26.127488 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:26.127496 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:26.127499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:26.127722 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:26.127912 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:09:26.127912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:09:26.127912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:09:26.128052 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:09:26.128372 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:09:26.128380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:09:26.128519 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:09:26.128608 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:09:26.128735 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260812 21:09:26.129359 31786 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43447
I20260812 21:09:26.129449 31786 heartbeater.cc:464] Registering TS with master...
I20260812 21:09:26.129629 31786 heartbeater.cc:511] Master 127.11.207.190:43447 requested a full tablet report, sending...
I20260812 21:09:26.130010 30922 ts_manager.cc:194] Registered new tserver with Master: f646c8217d07490dafde4373d676eb40 (127.11.207.131:44343)
I20260812 21:09:26.130393 30922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:35629
I20260812 21:09:26.139256 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:09:26.146495 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.140570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.11.207.190:43447)
0812 21:09:26.140638 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.140640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.140713 (+ 73us) server_negotiation.cc:407] Connection header received
0812 21:09:26.140785 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.140786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.140798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.140844 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:09:26.141482 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.141998 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.142806 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.143110 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.143176 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.144434 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:09:26.144436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:09:26.144436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:09:26.144444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:09:26.145575 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:26.145878 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:26.145879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:26.145879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:26.145933 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:09:26.146064 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:09:26.146065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:09:26.146065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:09:26.146210 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:09:26.146286 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.146366 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.146428 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:09:26.147930 30922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:09:26.149633 30922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:09:26.156277 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.151975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45821 (local address 127.0.0.1:35594)
0812 21:09:26.152415 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.152459 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:26.152520 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:26.153012 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:26.153013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:26.153014 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:26.153048 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.153049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.154363 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.154364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.155354 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.155357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.156010 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.156011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.156065 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.156084 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 21:09:26.156131 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:09:26.156183 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260812 21:09:26.156332 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.152168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.11.207.130:36933)
0812 21:09:26.152418 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.152423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.152682 (+ 259us) server_negotiation.cc:407] Connection header received
0812 21:09:26.152731 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.152733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.152751 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.152831 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:26.153114 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.153835 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.155142 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.155889 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.155965 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.156024 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.156180 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.156234 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
I20260812 21:09:26.156499 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.152263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35594 (local address 127.11.207.129:45821)
0812 21:09:26.152836 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.152841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.152850 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:09:26.152891 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.152893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.152907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.153011 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:26.153094 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.154250 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.155431 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.155916 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.156239 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.156322 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.156373 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.156418 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":96,"threads_started":1}
I20260812 21:09:26.156544 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.151975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:36933 (local address 127.0.0.1:47840)
0812 21:09:26.152590 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.152596 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:26.152632 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:26.152930 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:26.152932 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:26.152937 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:26.153024 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.153026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.153931 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.153932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.155056 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.155059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.155978 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.155980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.156065 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.156084 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 21:09:26.156155 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 21:09:26.156190 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":7,"thread_start_us":97,"threads_started":1}
I20260812 21:09:26.156682 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.151984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44343 (local address 127.0.0.1:44170)
0812 21:09:26.152674 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.152680 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:26.152715 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:26.152927 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:26.152932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:26.152934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:26.153034 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.153036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.154401 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.154402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.155334 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.155337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.156460 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.156462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.156525 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.156530 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:09:26.156578 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:09:26.156615 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":5,"thread_start_us":110,"threads_started":1}
I20260812 21:09:26.157042 31812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.152255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.11.207.131:44343)
0812 21:09:26.152493 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.152495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.152744 (+ 249us) server_negotiation.cc:407] Connection header received
0812 21:09:26.152794 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.152797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.152813 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.153122 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:26.153197 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.154287 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.155445 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.156103 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.156169 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.156236 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.156803 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.156845 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":106,"threads_started":1}
I20260812 21:09:26.157200 31173 tablet_service.cc:1467] Processing CreateTablet for tablet 09e5d34daa4e4945b270a93154814570 (DEFAULT_TABLE table=test-table [id=2931670ce6874f6b84fc84347601b384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:26.157238 31413 tablet_service.cc:1467] Processing CreateTablet for tablet 09e5d34daa4e4945b270a93154814570 (DEFAULT_TABLE table=test-table [id=2931670ce6874f6b84fc84347601b384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:26.157591 31173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e5d34daa4e4945b270a93154814570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:26.157591 31413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e5d34daa4e4945b270a93154814570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:26.161085 31814 tablet_bootstrap.cc:492] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Bootstrap starting.
I20260812 21:09:26.161067 31648 tablet_service.cc:1467] Processing CreateTablet for tablet 09e5d34daa4e4945b270a93154814570 (DEFAULT_TABLE table=test-table [id=2931670ce6874f6b84fc84347601b384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:09:26.161475 31648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e5d34daa4e4945b270a93154814570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:09:26.161805 31814 tablet_bootstrap.cc:654] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:26.162382 31814 log.cc:824] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:26.162775 31816 tablet_bootstrap.cc:492] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: Bootstrap starting.
I20260812 21:09:26.163162 31814 tablet_bootstrap.cc:492] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: No bootstrap required, opened a new log
I20260812 21:09:26.163213 31814 ts_tablet_manager.cc:1397] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:09:26.163457 31816 tablet_bootstrap.cc:654] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:26.164047 31816 log.cc:824] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:26.164918 31816 tablet_bootstrap.cc:492] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: No bootstrap required, opened a new log
I20260812 21:09:26.164989 31821 tablet_bootstrap.cc:492] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Bootstrap starting.
I20260812 21:09:26.165009 31814 raft_consensus.cc:348] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.165130 31816 ts_tablet_manager.cc:1397] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:09:26.165175 31814 raft_consensus.cc:374] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:26.165220 31814 raft_consensus.cc:729] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fd96a5dcfd34be8bb499ae4d56c83d4, State: Initialized, Role: FOLLOWER
I20260812 21:09:26.165352 31814 consensus_queue.cc:260] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.165657 31814 ts_tablet_manager.cc:1428] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:09:26.165745 31821 tablet_bootstrap.cc:654] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Neither blocks nor log segments found. Creating new log.
I20260812 21:09:26.165830 31525 heartbeater.cc:503] Master 127.11.207.190:43447 was elected leader, sending a full tablet report...
I20260812 21:09:26.166334 31821 log.cc:824] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Log is configured to *not* fsync() on all Append() calls
I20260812 21:09:26.167153 31821 tablet_bootstrap.cc:492] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: No bootstrap required, opened a new log
I20260812 21:09:26.167222 31821 ts_tablet_manager.cc:1397] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:09:26.167318 31816 raft_consensus.cc:348] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.167475 31816 raft_consensus.cc:374] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:26.167497 31816 raft_consensus.cc:729] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9164bfdf6e7442f3ac757c3674932e02, State: Initialized, Role: FOLLOWER
I20260812 21:09:26.167618 31816 consensus_queue.cc:260] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.167887 31816 ts_tablet_manager.cc:1428] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:09:26.167924 31280 heartbeater.cc:503] Master 127.11.207.190:43447 was elected leader, sending a full tablet report...
I20260812 21:09:26.169473 31821 raft_consensus.cc:348] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.169598 31821 raft_consensus.cc:374] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:09:26.169618 31821 raft_consensus.cc:729] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f646c8217d07490dafde4373d676eb40, State: Initialized, Role: FOLLOWER
I20260812 21:09:26.169723 31821 consensus_queue.cc:260] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.170013 31821 ts_tablet_manager.cc:1428] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:09:26.170130 31786 heartbeater.cc:503] Master 127.11.207.190:43447 was elected leader, sending a full tablet report...
W20260812 21:09:26.179867 31526 tablet.cc:2369] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:09:26.264283 31281 tablet.cc:2369] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:09:26.370415 31787 tablet.cc:2369] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:09:26.421787 31827 raft_consensus.cc:482] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:09:26.421926 31827 raft_consensus.cc:504] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.422256 31827 leader_election.cc:288] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fd96a5dcfd34be8bb499ae4d56c83d4 (127.11.207.130:36933), f646c8217d07490dafde4373d676eb40 (127.11.207.131:44343)
I20260812 21:09:26.425186 31812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.422576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:38597 (local address 127.11.207.131:44343)
0812 21:09:26.422620 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.422622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.422671 (+ 49us) server_negotiation.cc:407] Connection header received
0812 21:09:26.422769 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.422770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.422778 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.422823 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:26.422971 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.423439 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.424413 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.424805 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.424868 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.424877 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.425084 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.425121 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:09:26.425220 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.422508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:39559 (local address 127.11.207.130:36933)
0812 21:09:26.422564 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.422567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.422763 (+ 196us) server_negotiation.cc:407] Connection header received
0812 21:09:26.422816 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.422817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.422828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.422878 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:09:26.423024 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.423505 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.424518 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.424890 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.424954 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.424963 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.425113 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.425148 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:09:26.425243 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.422461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44343 (local address 127.11.207.129:38597)
0812 21:09:26.422614 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.422625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:26.422690 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:26.422839 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:26.422840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:26.422842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:26.422917 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.422919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.423514 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.423515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.424306 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.424308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.424882 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.424883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.425001 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.425006 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:09:26.425059 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:09:26.425103 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260812 21:09:26.425421 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.422449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:36933 (local address 127.11.207.129:39559)
0812 21:09:26.422678 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.422689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:09:26.422759 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:09:26.422888 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:09:26.422889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:09:26.422891 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:09:26.422963 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.422965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.423582 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.423583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.424433 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:09:26.424436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.424967 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.424968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:09:26.425049 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.425053 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:09:26.425098 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:09:26.425142 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":28,"thread_start_us":154,"threads_started":1}
I20260812 21:09:26.425596 31688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e5d34daa4e4945b270a93154814570" candidate_uuid: "9164bfdf6e7442f3ac757c3674932e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f646c8217d07490dafde4373d676eb40" is_pre_election: true
I20260812 21:09:26.425770 31688 raft_consensus.cc:2454] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9164bfdf6e7442f3ac757c3674932e02 in term 0.
I20260812 21:09:26.425858 31445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e5d34daa4e4945b270a93154814570" candidate_uuid: "9164bfdf6e7442f3ac757c3674932e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" is_pre_election: true
I20260812 21:09:26.426035 31070 leader_election.cc:302] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9164bfdf6e7442f3ac757c3674932e02, f646c8217d07490dafde4373d676eb40; no voters:
I20260812 21:09:26.426151 31445 raft_consensus.cc:2454] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9164bfdf6e7442f3ac757c3674932e02 in term 0.
I20260812 21:09:26.426200 31827 raft_consensus.cc:2781] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:09:26.426263 31827 raft_consensus.cc:482] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:09:26.426296 31827 raft_consensus.cc:3037] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:26.427058 31827 raft_consensus.cc:504] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.427181 31827 leader_election.cc:288] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [CANDIDATE]: Term 1 election: Requested vote from peers 6fd96a5dcfd34be8bb499ae4d56c83d4 (127.11.207.130:36933), f646c8217d07490dafde4373d676eb40 (127.11.207.131:44343)
I20260812 21:09:26.427372 31445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e5d34daa4e4945b270a93154814570" candidate_uuid: "9164bfdf6e7442f3ac757c3674932e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4"
I20260812 21:09:26.427372 31688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e5d34daa4e4945b270a93154814570" candidate_uuid: "9164bfdf6e7442f3ac757c3674932e02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f646c8217d07490dafde4373d676eb40"
I20260812 21:09:26.427454 31445 raft_consensus.cc:3037] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:26.427455 31688 raft_consensus.cc:3037] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:09:26.428552 31688 raft_consensus.cc:2454] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9164bfdf6e7442f3ac757c3674932e02 in term 1.
I20260812 21:09:26.428552 31445 raft_consensus.cc:2454] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9164bfdf6e7442f3ac757c3674932e02 in term 1.
I20260812 21:09:26.428745 31076 leader_election.cc:302] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fd96a5dcfd34be8bb499ae4d56c83d4, 9164bfdf6e7442f3ac757c3674932e02; no voters:
I20260812 21:09:26.428838 31827 raft_consensus.cc:2781] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:09:26.428997 31827 raft_consensus.cc:686] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [term 1 LEADER]: Becoming Leader. State: Replica: 9164bfdf6e7442f3ac757c3674932e02, State: Running, Role: LEADER
I20260812 21:09:26.429092 31827 consensus_queue.cc:237] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } }
I20260812 21:09:26.429998 30921 catalog_manager.cc:5495] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9164bfdf6e7442f3ac757c3674932e02 (127.11.207.129). New cstate: current_term: 1 leader_uuid: "9164bfdf6e7442f3ac757c3674932e02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9164bfdf6e7442f3ac757c3674932e02" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 } health_report { overall_health: UNKNOWN } } }
I20260812 21:09:26.442574 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.440402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.11.207.129:45821)
0812 21:09:26.440444 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:09:26.440446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:09:26.440477 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:09:26.440548 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:09:26.440549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:09:26.440567 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:09:26.440630 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:09:26.440806 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.441292 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.441975 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:09:26.442158 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:09:26.442199 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:09:26.442327 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:09:26.442424 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:09:26.442479 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:09:26.442492 (+ 13us) server_negotiation.cc:299] Negotiation successful
0812 21:09:26.442510 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:09:26.444550 31688 raft_consensus.cc:1261] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 1 FOLLOWER]: Refusing update from remote peer 9164bfdf6e7442f3ac757c3674932e02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:26.444550 31445 raft_consensus.cc:1261] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 1 FOLLOWER]: Refusing update from remote peer 9164bfdf6e7442f3ac757c3674932e02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:09:26.444977 31834 consensus_queue.cc:1035] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f646c8217d07490dafde4373d676eb40" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:26.445116 31827 consensus_queue.cc:1035] T 09e5d34daa4e4945b270a93154814570 P 9164bfdf6e7442f3ac757c3674932e02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fd96a5dcfd34be8bb499ae4d56c83d4" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:09:26.447299 31842 mvcc.cc:205] Tried to move back new op lower bound from 7317786486554079232 to 7317786486494195712. Current Snapshot: MvccSnapshot[applied={T|T < 7317786486554079232}]
I20260812 21:09:26.447576 31844 mvcc.cc:205] Tried to move back new op lower bound from 7317786486554079232 to 7317786486494195712. Current Snapshot: MvccSnapshot[applied={T|T < 7317786486554079232}]
I20260812 21:09:26.448905 31846 mvcc.cc:205] Tried to move back new op lower bound from 7317786486554079232 to 7317786486494195712. Current Snapshot: MvccSnapshot[applied={T|T < 7317786486554079232 or (T in {7317786486554079232})}]
I20260812 21:09:26.449810 30921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:26.449909 30921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:09:26.450675 30921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c7eacc949f14b4987c5086505bece6d: Sending DeleteTablet for 3 replicas of tablet 09e5d34daa4e4945b270a93154814570
I20260812 21:09:26.451051 31413 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e5d34daa4e4945b270a93154814570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:26 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:47840
I20260812 21:09:26.451051 31173 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e5d34daa4e4945b270a93154814570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:26 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:35594
I20260812 21:09:26.451141 31648 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e5d34daa4e4945b270a93154814570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:09:26 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:44170
I20260812 21:09:26.451289 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 31043
I20260812 21:09:26.456295 31856 tablet_replica.cc:330] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: stopping tablet replica
I20260812 21:09:26.456429 31856 raft_consensus.cc:2229] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:26.456548 31856 raft_consensus.cc:2258] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:26.456717 31860 tablet_replica.cc:330] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: stopping tablet replica
I20260812 21:09:26.456805 31860 raft_consensus.cc:2229] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:09:26.456892 31860 raft_consensus.cc:2258] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:09:26.461580 31860 ts_tablet_manager.cc:1905] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:26.461580 31856 ts_tablet_manager.cc:1905] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:09:26.463452 31860 ts_tablet_manager.cc:1918] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:26.463452 31856 ts_tablet_manager.cc:1918] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:09:26.463538 31860 log.cc:1196] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/09e5d34daa4e4945b270a93154814570
I20260812 21:09:26.463647 31856 log.cc:1196] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/09e5d34daa4e4945b270a93154814570
I20260812 21:09:26.463824 31860 ts_tablet_manager.cc:1939] T 09e5d34daa4e4945b270a93154814570 P f646c8217d07490dafde4373d676eb40: Deleting consensus metadata
I20260812 21:09:26.464028 31856 ts_tablet_manager.cc:1939] T 09e5d34daa4e4945b270a93154814570 P 6fd96a5dcfd34be8bb499ae4d56c83d4: Deleting consensus metadata
I20260812 21:09:26.464485 30897 catalog_manager.cc:4841] TS 6fd96a5dcfd34be8bb499ae4d56c83d4 (127.11.207.130:36933): tablet 09e5d34daa4e4945b270a93154814570 (table test-table [id=2931670ce6874f6b84fc84347601b384]) successfully deleted
I20260812 21:09:26.464676 30893 catalog_manager.cc:4841] TS f646c8217d07490dafde4373d676eb40 (127.11.207.131:44343): tablet 09e5d34daa4e4945b270a93154814570 (table test-table [id=2931670ce6874f6b84fc84347601b384]) successfully deleted
W20260812 21:09:26.474004 30894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:45821 (error 108)
I20260812 21:09:26.474182 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 31291
I20260812 21:09:26.476702 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:09:26.476558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45821 (local address 127.0.0.1:35610)
0812 21:09:26.476617 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:09:26.476662 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:45821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 21:09:26.476837 30894 catalog_manager.cc:4584] TS 9164bfdf6e7442f3ac757c3674932e02 (127.11.207.129:45821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09e5d34daa4e4945b270a93154814570: Network error: Client connection negotiation failed: client connection to 127.11.207.129:45821: connect: Connection refused (error 111)
I20260812 21:09:26.482968 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 31534
I20260812 21:09:26.497874 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 30865
2026-08-12T21:09:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2427 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:09:26 dist-test-slave-lkx2 krb5kdc[31869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:09:26 dist-test-slave-lkx2 krb5kdc[31869](info): set up 2 sockets
Aug 12 21:09:26 dist-test-slave-lkx2 krb5kdc[31869](info): commencing operation
krb5kdc: starting...
W20260812 21:09:28.050119 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.527s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:09:28 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:09:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/rangeradmin_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/rangeradmin_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/rangerlookup_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/rangerlookup_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
I20260812 21:09:28.352878 12094 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwFLJxB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/postgres -l logfile start
2026-08-12 21:09:30.855 UTC [31934] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 21:09:30.855 UTC [31934] LOG: listening on IPv4 address "127.11.207.148", port 43665
2026-08-12 21:09:30.858 UTC [31934] LOG: listening on Unix socket "/tmp/.s.PGSQL.43665"
2026-08-12 21:09:30.887 UTC [31938] LOG: database system was shut down at 2026-08-12 21:09:29 UTC
2026-08-12 21:09:30.891 UTC [31934] LOG: database system is ready to accept connections
I20260812 21:09:32.000437 12094 mini_postgres.cc:96] Postgres bound to 43665
2026-08-12 21:09:32.009 UTC [31952] FATAL: database "slave" does not exist
127.11.207.148:43665 - accepting connections
I20260812 21:09:32.010108 12094 mini_ranger.cc:161] Starting Ranger...
I20260812 21:09:32.027393 12094 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 21:09:32.486222 12094 mini_ranger.cc:87] Created ranger Postgres database
I20260812 21:09:32.486296 12094 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-admin
2026-08-12 21:09:32,902 [I] DB FLAVOR :POSTGRES
2026-08-12 21:09:32,902 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:09:32,902 [I] Checking connection..
2026-08-12 21:09:32,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 21:09:33,192 [I] Checking connection passed.
2026-08-12 21:09:33,193 [I] --------- Verifying version history table ---------
2026-08-12 21:09:33,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:09:33,488 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:09:33,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:09:33,761 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:09:33,761 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 21:09:33,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 21:09:34.054 UTC [32086] WARNING: there is no transaction in progress
2026-08-12 21:09:34,065 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 21:09:34,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:09:34,360 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 21:09:34,360 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:09:34,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 21:09:34,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 21:09:34,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lkx2.c.gcp-upstream.internal','N') ;"
2026-08-12 21:09:35,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:09:35,483 [I] Table x_portal_user does not exist in database ranger
2026-08-12 21:09:35,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:09:35,763 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 21:09:35,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:09:36,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:09:36,302 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 21:09:36,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 21:09:37.270 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.286 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.302 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.317 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.333 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.350 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.405 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.412 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.426 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.436 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.446 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.460 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.469 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.475 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.481 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:37.487 UTC [32295] WARNING: there is no transaction in progress
2026-08-12 21:09:38,115 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 21:09:38,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 21:09:38,372 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 21:09:38,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:09:38,650 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 21:09:38,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:09:38,914 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 21:09:38,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:09:39,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:09:39,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lkx2.c.gcp-upstream.internal';"
2026-08-12 21:09:39,683 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 21:09:39,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 21:09:39,937 [I] DB_PATCHES have already been applied
I20260812 21:09:39.942924 12094 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwFLJxB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 21:09:39.942994 12094 mini_ranger.cc:193] Using host: 127.11.207.148
I20260812 21:09:39.947047 12094 mini_ranger.cc:227] Ranger admin URL: http://127.11.207.148:46355
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 9:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 21:09:41.233232 12094 mini_ranger.cc:160] Time spent starting Ranger: real 9.223s user 0.000s sys 0.011s
Aug 12, 2026 9:10:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 9:10:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 9:10:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 9:10:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 9:10:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 21:10:03.817191 12094 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/rangerkms_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/rangerkms_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.207.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.207.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 21:10:03 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569003, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 21:10:03.907933 12094 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 21:10:03.922715 12094 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 21:10:04.831620 12094 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 21:10:04.831744 12094 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms
I20260812 21:10:04.831760 12094 mini_ranger_kms.cc:227] Using postgres at 127.11.207.148:43665
2026-08-12 21:10:05,250 [I] DB FLAVOR :POSTGRES
2026-08-12 21:10:05,250 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:10:05,251 [I] Checking connection
2026-08-12 21:10:05,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 21:10:05,595 [I] connection success
2026-08-12 21:10:05,595 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 21:10:05,595 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 21:10:05,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 21:10:05,879 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 21:10:05,880 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:10:05,880 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 21:10:05,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 21:10:06,189 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 21:10:06.693112 12094 mini_ranger_kms.cc:310] Created kms service
I20260812 21:10:06.898973 12094 mini_ranger_kms.cc:326] Created kudu user
I20260812 21:10:06.948937 12094 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 21:10:06 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569003, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:10:07.382279 12094 mini_ranger_kms.cc:384] Added ranger policy
I20260812 21:10:07.382409 12094 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 21:10:07.382429 12094 mini_ranger_kms.cc:240] Using host: 127.11.207.148
I20260812 21:10:07.386623 12094 mini_ranger_kms.cc:276] Ranger KMS PID: 32667
I20260812 21:10:07.386757 12094 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.207.148:52411
Aug 12, 2026 9:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 9:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 9:10:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 9:10:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 21:10:09 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786569009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.207.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:10:09.950600 12094 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.043s user 0.009s sys 0.385s
I20260812 21:10:09.950718 12094 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 21:10:09.950816 12094 mini_ranger_kms.cc:401] 127.11.207.148:52411/kms/v1/keys
Aug 12 21:10:10 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.207.148@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
Aug 12 21:10:10 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.207.148@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
Aug 12 21:10:13 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:10:13.550822 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:35967
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:45139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:35967
--ranger_config_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:52411/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:10:13.647742 302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:10:13.647984 302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:10:13.648017 302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:10:13.649298 302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:10:13.649345 302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:10:13.649365 302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:10:13.649380 302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:10:13.649396 302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:10:13.651374 302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:45139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.207.148:52411/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:35967
--ranger_config_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:10:13.651710 302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:10:13.651953 302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:10:13.658353 311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:13.658497 318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:13.658383 316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:10:13.658704 302 server_base.cc:1034] running on GCE node
I20260812 21:10:13.658988 302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:10:13.659281 302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:10:13.660410 302 hybrid_clock.cc:648] HybridClock initialized: now 1786569013660383 us; error 41 us; skew 500 ppm
Aug 12 21:10:13 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:10:13.663036 302 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:10:13.663489 302 webserver.cc:460] Webserver started at http://127.11.207.190:44623/ using document root <none> and password file <none>
I20260812 21:10:13.663722 302 fs_manager.cc:359] Metadata directory not provided
I20260812 21:10:13.663761 302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:10:13.663877 302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:10:13 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:10:13.716003 302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "5ccc7c2ab121479babcc224a8509fa9c"
format_stamp: "Formatted at 2026-08-12 21:10:13 on dist-test-slave-lkx2"
server_key: "1453e351237e5f45a0b31db1a35dd329"
server_key_iv: "1c55f88b2cda3f121593224af9deded1"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:13.716516 302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "5ccc7c2ab121479babcc224a8509fa9c"
format_stamp: "Formatted at 2026-08-12 21:10:13 on dist-test-slave-lkx2"
server_key: "1453e351237e5f45a0b31db1a35dd329"
server_key_iv: "1c55f88b2cda3f121593224af9deded1"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:13.718514 302 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 21:10:13.733893 331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:13.734140 302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260812 21:10:13.734265 302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "5ccc7c2ab121479babcc224a8509fa9c"
format_stamp: "Formatted at 2026-08-12 21:10:13 on dist-test-slave-lkx2"
server_key: "1453e351237e5f45a0b31db1a35dd329"
server_key_iv: "1c55f88b2cda3f121593224af9deded1"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:13.734372 302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:10:13.763031 302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:10:13.765054 302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:10:13.765262 302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:10:13.771957 302 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:35967
I20260812 21:10:13.772166 444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:35967 every 8 connection(s)
I20260812 21:10:13.772475 302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:10:13.773422 447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:10:13.776111 447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c: Bootstrap starting.
I20260812 21:10:13.776775 447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c: Neither blocks nor log segments found. Creating new log.
I20260812 21:10:13.777194 447 log.cc:824] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:10:13.777915 447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c: No bootstrap required, opened a new log
I20260812 21:10:13.778717 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 302
I20260812 21:10:13.778815 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:10:13.779001 12094 external_mini_cluster.cc:1428] Setting key 3e79c97b0954756f8a99379b8977f903
I20260812 21:10:13.784675 447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccc7c2ab121479babcc224a8509fa9c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35967 } }
I20260812 21:10:13.784974 447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:10:13.785023 447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccc7c2ab121479babcc224a8509fa9c, State: Initialized, Role: FOLLOWER
I20260812 21:10:13.785256 447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccc7c2ab121479babcc224a8509fa9c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35967 } }
I20260812 21:10:13.785408 447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:10:13.785465 447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:10:13.785533 447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:10:13.786571 447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccc7c2ab121479babcc224a8509fa9c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35967 } }
I20260812 21:10:13.786708 447 leader_election.cc:302] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ccc7c2ab121479babcc224a8509fa9c; no voters:
I20260812 21:10:13.786888 447 leader_election.cc:288] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:10:13.787185 447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:10:13.789605 447 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 21:10:13.790490 458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:10:13.790617 458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [term 1 LEADER]: Becoming Leader. State: Replica: 5ccc7c2ab121479babcc224a8509fa9c, State: Running, Role: LEADER
I20260812 21:10:13.790721 458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccc7c2ab121479babcc224a8509fa9c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35967 } }
I20260812 21:10:13.791051 458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ccc7c2ab121479babcc224a8509fa9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccc7c2ab121479babcc224a8509fa9c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35967 } } }
I20260812 21:10:13.791123 458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [sys.catalog]: This master's current role is: LEADER
I20260812 21:10:13.791230 458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ccc7c2ab121479babcc224a8509fa9c. Latest consensus state: current_term: 1 leader_uuid: "5ccc7c2ab121479babcc224a8509fa9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ccc7c2ab121479babcc224a8509fa9c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35967 } } }
I20260812 21:10:13.791275 458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c [sys.catalog]: This master's current role is: LEADER
I20260812 21:10:13.791406 488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:10:13.792128 488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:10:13.793834 488 catalog_manager.cc:1347] Generated new cluster ID: c0d8cdfe6e6248c5b2f288e4d053e7b1
I20260812 21:10:13.794174 488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:10:13 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:10:13.809648 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:13.781614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.11.207.190:35967)
0812 21:10:13.781843 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:13.781854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:13.788715 (+ 6861us) server_negotiation.cc:407] Connection header received
0812 21:10:13.788892 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:13.788901 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:13.788973 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:13.789131 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:10:13.790130 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:13.791664 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:13.792727 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:13.793351 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:13.793485 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:13.803090 (+ 9605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:10:13.803199 (+ 109us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:10:13.803199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:10:13.803333 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:10:13.806544 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:13.806984 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:13.806988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:13.806988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:13.807103 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:13.807286 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:13.807287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:13.807287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:13.807697 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:10:13.807894 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:13.808465 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 21:10:13.808913 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":96,"threads_started":1}
I20260812 21:10:13.811275 488 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:10:13.812476 488 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:10:13.816310 488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c: Generated new TSK 0
I20260812 21:10:13.816612 488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:10:14.619047 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:52411/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:35967
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:45139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:10:14.752866 540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:10:14.753079 540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:10:14.753114 540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:10:14.754386 540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:10:14.754432 540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:10:14.754469 540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:10:14.756186 540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:45139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.207.148:52411/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:10:14.756461 540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:10:14.756688 540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:10:14.763036 549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:14.763036 548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:14.763161 553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:10:14.763314 540 server_base.cc:1034] running on GCE node
I20260812 21:10:14.763561 540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:10:14.763816 540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:10:14.764966 540 hybrid_clock.cc:648] HybridClock initialized: now 1786569014764949 us; error 36 us; skew 500 ppm
Aug 12 21:10:14 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:10:14.767576 540 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:10:14.768043 540 webserver.cc:460] Webserver started at http://127.11.207.129:44245/ using document root <none> and password file <none>
I20260812 21:10:14.768266 540 fs_manager.cc:359] Metadata directory not provided
I20260812 21:10:14.768316 540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:10:14.768481 540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:10:14 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:10:14.783522 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "e9a78ecedf7f44e7984d974be4369eb0"
format_stamp: "Formatted at 2026-08-12 21:10:14 on dist-test-slave-lkx2"
server_key: "9d0db46ffb6a3eaad694ab752bfdb315"
server_key_iv: "db9fb982c5a3f6ad811e804c0114df91"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:14.784190 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "e9a78ecedf7f44e7984d974be4369eb0"
format_stamp: "Formatted at 2026-08-12 21:10:14 on dist-test-slave-lkx2"
server_key: "9d0db46ffb6a3eaad694ab752bfdb315"
server_key_iv: "db9fb982c5a3f6ad811e804c0114df91"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:14.786283 540 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 21:10:14.799396 565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:14.799630 540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:10:14.799744 540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "e9a78ecedf7f44e7984d974be4369eb0"
format_stamp: "Formatted at 2026-08-12 21:10:14 on dist-test-slave-lkx2"
server_key: "9d0db46ffb6a3eaad694ab752bfdb315"
server_key_iv: "db9fb982c5a3f6ad811e804c0114df91"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:14.799841 540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:10:14.812816 540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:10:14.815371 540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:10:14.815586 540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:10:14.815923 540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:10:14.816542 540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:10:14.816648 540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:14.816782 540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:10:14.816857 540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:14.827831 540 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:40507
I20260812 21:10:14.828450 540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:10:14.828861 791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:40507 every 8 connection(s)
I20260812 21:10:14.837742 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 540
Aug 12 21:10:14 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:10:14.837863 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:10:14.838088 12094 external_mini_cluster.cc:1428] Setting key b7279e45d1401480fcbe815f01d7993f
I20260812 21:10:14.842440 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:14.829935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:38153 (local address 127.11.207.190:35967)
0812 21:10:14.830050 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:14.830052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:14.830205 (+ 153us) server_negotiation.cc:407] Connection header received
0812 21:10:14.831105 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:14.831107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:14.831121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:14.831197 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:10:14.832109 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:14.832953 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:14.834542 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:14.834874 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:14.834953 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:14.838341 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:10:14.838344 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:10:14.838344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:10:14.838353 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:10:14.840592 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:14.841175 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:14.841177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:14.841177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:14.841257 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:14.841606 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:14.841607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:14.841607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:14.841779 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:10:14.841860 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:14.842219 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 21:10:14.842316 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260812 21:10:14.842700 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:14.829834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:35967 (local address 127.11.207.129:38153)
0812 21:10:14.830030 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:14.830087 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:14.830955 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:14.831289 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:14.831294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:14.831643 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:10:14.831976 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:14.831984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:14.833075 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:14.833076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:14.834432 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:14.834436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:14.834986 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:14.834987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:14.835146 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:14.835863 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:10:14.835875 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:10:14.838206 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:10:14.840726 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:10:14.840735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:10:14.840738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:10:14.841078 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:10:14.841341 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:10:14.841342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:10:14.841343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:10:14.841518 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:10:14.841858 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:10:14.841865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:10:14.842054 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 21:10:14.842198 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 21:10:14.842386 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260812 21:10:14.843420 797 heartbeater.cc:347] Connected to a master server at 127.11.207.190:35967
I20260812 21:10:14.843542 797 heartbeater.cc:464] Registering TS with master...
I20260812 21:10:14.843771 797 heartbeater.cc:511] Master 127.11.207.190:35967 requested a full tablet report, sending...
I20260812 21:10:14.844472 354 ts_manager.cc:194] Registered new tserver with Master: e9a78ecedf7f44e7984d974be4369eb0 (127.11.207.129:40507)
I20260812 21:10:14.845386 354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:38153
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:10:14.873463 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:52411/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:35967
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:45139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:10:14 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786569014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 21:10:15.013741 808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:10:15.014129 808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:10:15.014219 808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:10:15.016161 808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:10:15.016230 808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:10:15.016271 808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:10:15.018807 808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:45139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.207.148:52411/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:10:15.019312 808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:10:15.019702 808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:10:15.027606 826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:15.027632 827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:15.027791 830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:10:15.028126 808 server_base.cc:1034] running on GCE node
I20260812 21:10:15.028430 808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:10:15.028744 808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:10:15.029909 808 hybrid_clock.cc:648] HybridClock initialized: now 1786569015029886 us; error 37 us; skew 500 ppm
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:10:15.033914 808 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:10:15.034466 808 webserver.cc:460] Webserver started at http://127.11.207.130:38937/ using document root <none> and password file <none>
I20260812 21:10:15.034765 808 fs_manager.cc:359] Metadata directory not provided
I20260812 21:10:15.034827 808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:10:15.034965 808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:10:15.053920 808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "f4bca3dcbe084701be54e2a3da7a54f9"
format_stamp: "Formatted at 2026-08-12 21:10:15 on dist-test-slave-lkx2"
server_key: "efe180ee34cd7ad8c636555976c70d57"
server_key_iv: "b39390896d70324139d5cc83efa888d1"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:15.054391 808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "f4bca3dcbe084701be54e2a3da7a54f9"
format_stamp: "Formatted at 2026-08-12 21:10:15 on dist-test-slave-lkx2"
server_key: "efe180ee34cd7ad8c636555976c70d57"
server_key_iv: "b39390896d70324139d5cc83efa888d1"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:15.057384 808 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 21:10:15.078003 838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:15.078330 808 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260812 21:10:15.078455 808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "f4bca3dcbe084701be54e2a3da7a54f9"
format_stamp: "Formatted at 2026-08-12 21:10:15 on dist-test-slave-lkx2"
server_key: "efe180ee34cd7ad8c636555976c70d57"
server_key_iv: "b39390896d70324139d5cc83efa888d1"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:15.078572 808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:10:15.097886 808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:10:15.100414 808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:10:15.100597 808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:10:15.101033 808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:10:15.101498 808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:10:15.101557 808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:15.101589 808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:10:15.101658 808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:15.111514 808 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:38349
I20260812 21:10:15.111750 1048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:38349 every 8 connection(s)
I20260812 21:10:15.112056 808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:10:15.116283 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 808
I20260812 21:10:15.116412 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:10:15.116580 12094 external_mini_cluster.cc:1428] Setting key c5cbaac41ee750f2ec1c7f735ced277d
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:10:15.124730 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.113186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:47845 (local address 127.11.207.190:35967)
0812 21:10:15.113246 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.113248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.113547 (+ 299us) server_negotiation.cc:407] Connection header received
0812 21:10:15.114375 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.114376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.114390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.114434 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:10:15.115208 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.115773 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.117506 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.117768 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.117836 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.120815 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:10:15.120817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:10:15.120818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:10:15.120827 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:10:15.122787 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:15.123366 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:15.123367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:15.123368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:15.123446 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:15.123800 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:15.123802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:15.123802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:15.123963 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:10:15.124012 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.124485 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.124600 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:10:15.125584 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.113139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:35967 (local address 127.11.207.130:47845)
0812 21:10:15.113377 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.113443 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.114245 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.114522 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.114528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.114795 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:10:15.115101 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.115108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.115866 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.115867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.117370 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.117374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.117871 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.117873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.118031 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.118699 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:10:15.118711 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:10:15.120672 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:10:15.122900 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:10:15.122908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:10:15.122911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:10:15.123254 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:10:15.123539 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:10:15.123540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:10:15.123541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:10:15.123702 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:10:15.124065 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:10:15.124072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:10:15.124263 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.125126 (+ 863us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.125333 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":141,"threads_started":1}
I20260812 21:10:15.126399 1049 heartbeater.cc:347] Connected to a master server at 127.11.207.190:35967
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
I20260812 21:10:15.126510 1049 heartbeater.cc:464] Registering TS with master...
I20260812 21:10:15.126724 1049 heartbeater.cc:511] Master 127.11.207.190:35967 requested a full tablet report, sending...
I20260812 21:10:15.127308 354 ts_manager.cc:194] Registered new tserver with Master: f4bca3dcbe084701be54e2a3da7a54f9 (127.11.207.130:38349)
I20260812 21:10:15.127869 354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:47845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:10:15.148638 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:52411/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:35967
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:45139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:10:15.258605 1058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:10:15.258838 1058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:10:15.258875 1058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:10:15.260632 1058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:10:15.260757 1058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:10:15.260864 1058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:10:15.263278 1058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:45139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.207.148:52411/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:35967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:10:15.263554 1058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:10:15.263901 1058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:10:15.272370 1076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:10:15.272688 1072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:10:15.272853 1058 server_base.cc:1034] running on GCE node
W20260812 21:10:15.273097 1070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:10:15.273433 1058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:10:15.273759 1058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:10:15.277118 1058 hybrid_clock.cc:648] HybridClock initialized: now 1786569015277083 us; error 54 us; skew 500 ppm
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:10:15.286473 1058 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:10:15.287226 1058 webserver.cc:460] Webserver started at http://127.11.207.131:42501/ using document root <none> and password file <none>
I20260812 21:10:15.287496 1058 fs_manager.cc:359] Metadata directory not provided
I20260812 21:10:15.287559 1058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:10:15.287735 1058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:10:15.303794 1058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "4ba4fb05ecb34709b67017458fb3592e"
format_stamp: "Formatted at 2026-08-12 21:10:15 on dist-test-slave-lkx2"
server_key: "6b4f05b932505fc574437180e5819dbc"
server_key_iv: "15d5bfad6739e61861659619ad59a2bb"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:15.304558 1058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "4ba4fb05ecb34709b67017458fb3592e"
format_stamp: "Formatted at 2026-08-12 21:10:15 on dist-test-slave-lkx2"
server_key: "6b4f05b932505fc574437180e5819dbc"
server_key_iv: "15d5bfad6739e61861659619ad59a2bb"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:15.306900 1058 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 21:10:15.323194 1095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:15.323472 1058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:10:15.323599 1058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "4ba4fb05ecb34709b67017458fb3592e"
format_stamp: "Formatted at 2026-08-12 21:10:15 on dist-test-slave-lkx2"
server_key: "6b4f05b932505fc574437180e5819dbc"
server_key_iv: "15d5bfad6739e61861659619ad59a2bb"
server_key_version: "kuduclusterkey@0"
I20260812 21:10:15.323724 1058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:10:15.361790 1058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:10:15.364463 1058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:10:15.364676 1058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:10:15.364985 1058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:10:15.365384 1058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:10:15.365425 1058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:15.365455 1058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:10:15.365477 1058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:10:15.377130 1058 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:38499
I20260812 21:10:15.377763 1058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:10:15.378127 1305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:38499 every 8 connection(s)
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569015, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:10:15.386672 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 1058
I20260812 21:10:15.386767 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:10:15.386991 12094 external_mini_cluster.cc:1428] Setting key 41652f93187a75ef5e695baacfabb796
I20260812 21:10:15.388855 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.378839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:58641 (local address 127.11.207.190:35967)
0812 21:10:15.378890 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.378893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.379157 (+ 264us) server_negotiation.cc:407] Connection header received
0812 21:10:15.379675 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.379676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.379695 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.379746 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:10:15.380411 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.380917 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.381962 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.382260 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.382305 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.385235 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:10:15.385237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:10:15.385237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:10:15.385245 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:10:15.386864 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:15.387739 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:15.387740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:15.387741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:15.387805 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:15.388099 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:15.388101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:15.388101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:15.388224 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:10:15.388282 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.388659 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.388749 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:10:15.389689 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.378835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:35967 (local address 127.11.207.131:58641)
0812 21:10:15.379014 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.379059 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.379573 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.379828 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.379833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.380026 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:10:15.380291 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.380297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.381000 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.381001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.381857 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.381860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.382406 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.382406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.382555 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.383161 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:10:15.383172 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:10:15.385119 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:10:15.387229 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:10:15.387237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:10:15.387240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:10:15.387647 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:10:15.387890 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:10:15.387891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:10:15.387892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:10:15.388032 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:10:15.388286 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:10:15.388293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:10:15.388486 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.389217 (+ 731us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.389391 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260812 21:10:15.390199 1308 heartbeater.cc:347] Connected to a master server at 127.11.207.190:35967
I20260812 21:10:15.390296 1308 heartbeater.cc:464] Registering TS with master...
I20260812 21:10:15.390480 1308 heartbeater.cc:511] Master 127.11.207.190:35967 requested a full tablet report, sending...
I20260812 21:10:15.390836 354 ts_manager.cc:194] Registered new tserver with Master: 4ba4fb05ecb34709b67017458fb3592e (127.11.207.131:38499)
I20260812 21:10:15.391198 354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:58641
I20260812 21:10:15.398718 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:10:15.407297 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.400214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.11.207.190:35967)
0812 21:10:15.400256 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.400258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.400550 (+ 292us) server_negotiation.cc:407] Connection header received
0812 21:10:15.400643 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.400644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.400655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.400686 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:10:15.401482 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.402201 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.403007 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.403287 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.403356 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.404698 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:10:15.404699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:10:15.404700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:10:15.404707 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:10:15.406259 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:15.406602 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:15.406603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:15.406604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:15.406671 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:10:15.406837 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:10:15.406837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:10:15.406838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:10:15.406948 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:10:15.407035 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.407130 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.407207 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:10:15.409121 354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:10:15.410993 354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:10:15.419793 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.413593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:38349 (local address 127.0.0.1:36678)
0812 21:10:15.414513 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.414555 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.414620 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.414913 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.414917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.414919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:10:15.414999 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.415001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.416635 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.416636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.417872 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.417876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.419497 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.419498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.419582 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.419595 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.419662 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.419715 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":145,"threads_started":1}
I20260812 21:10:15.420099 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.414204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.11.207.130:38349)
0812 21:10:15.414763 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.414767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.414775 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:10:15.414815 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.414816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.414828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.414893 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:10:15.415081 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.416191 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.417993 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.419209 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.419387 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.419458 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.419968 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.420010 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":151,"threads_started":1}
I20260812 21:10:15.421005 928 tablet_service.cc:1467] Processing CreateTablet for tablet b162e045a01b4bfe900bfd798284d0a0 (DEFAULT_TABLE table=test-table [id=e1d373f96c5546ba98331bc25f80b47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:10:15.421415 928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b162e045a01b4bfe900bfd798284d0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:10:15.422938 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.413593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40507 (local address 127.0.0.1:60122)
0812 21:10:15.415005 (+ 1412us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.415015 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.415063 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.415318 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.415319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.415321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:10:15.415361 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.415362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.420164 (+ 4802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.420165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.421542 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.421545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.422548 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.422705 (+ 157us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.422768 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.422773 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.422830 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.422870 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1395,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260812 21:10:15.423241 1337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.413860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.11.207.129:40507)
0812 21:10:15.414029 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.414034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.415123 (+ 1089us) server_negotiation.cc:407] Connection header received
0812 21:10:15.415163 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.415165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.415181 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.415250 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:10:15.415431 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.419660 (+ 4229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.421640 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.422376 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.422448 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.422509 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.423114 (+ 605us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.423165 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":58,"threads_started":1}
I20260812 21:10:15.424055 662 tablet_service.cc:1467] Processing CreateTablet for tablet b162e045a01b4bfe900bfd798284d0a0 (DEFAULT_TABLE table=test-table [id=e1d373f96c5546ba98331bc25f80b47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:10:15.424471 662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b162e045a01b4bfe900bfd798284d0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:10:15.426525 1343 tablet_bootstrap.cc:492] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Bootstrap starting.
I20260812 21:10:15.427675 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.413834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:38499 (local address 127.0.0.1:59050)
0812 21:10:15.420249 (+ 6415us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.420259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.420303 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.424051 (+ 3748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.424052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.424054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:10:15.424086 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.424087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.425373 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.425375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.426379 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.426382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.427410 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.427411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.427471 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.427475 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.427554 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.427594 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6404,"thread_start_us":119,"threads_started":1}
I20260812 21:10:15.427832 1343 tablet_bootstrap.cc:654] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Neither blocks nor log segments found. Creating new log.
I20260812 21:10:15.428184 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.414211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.11.207.131:38499)
0812 21:10:15.423782 (+ 9571us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.423787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.423799 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:10:15.423849 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.423851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.423869 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.423962 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:10:15.424165 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.425272 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.426482 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.427223 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.427317 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.427374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.428054 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.428099 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9506,"thread_start_us":87,"threads_started":1}
I20260812 21:10:15.428411 1343 log.cc:824] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:10:15.429019 1177 tablet_service.cc:1467] Processing CreateTablet for tablet b162e045a01b4bfe900bfd798284d0a0 (DEFAULT_TABLE table=test-table [id=e1d373f96c5546ba98331bc25f80b47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:10:15.429360 1177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b162e045a01b4bfe900bfd798284d0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:10:15.432255 1350 tablet_bootstrap.cc:492] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Bootstrap starting.
I20260812 21:10:15.433293 1346 tablet_bootstrap.cc:492] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: Bootstrap starting.
I20260812 21:10:15.434270 1346 tablet_bootstrap.cc:654] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: Neither blocks nor log segments found. Creating new log.
I20260812 21:10:15.434700 1346 log.cc:824] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: Log is configured to *not* fsync() on all Append() calls
I20260812 21:10:15.435523 1350 tablet_bootstrap.cc:654] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Neither blocks nor log segments found. Creating new log.
I20260812 21:10:15.435807 1346 tablet_bootstrap.cc:492] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: No bootstrap required, opened a new log
I20260812 21:10:15.435875 1346 ts_tablet_manager.cc:1397] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:10:15.436141 1350 log.cc:824] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:10:15.437180 1343 tablet_bootstrap.cc:492] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: No bootstrap required, opened a new log
I20260812 21:10:15.437258 1343 ts_tablet_manager.cc:1397] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.002s
I20260812 21:10:15.439111 1346 raft_consensus.cc:348] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.439292 1346 raft_consensus.cc:374] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:10:15.439327 1346 raft_consensus.cc:729] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a78ecedf7f44e7984d974be4369eb0, State: Initialized, Role: FOLLOWER
I20260812 21:10:15.439890 1346 consensus_queue.cc:260] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.439889 1343 raft_consensus.cc:348] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.440050 1343 raft_consensus.cc:374] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:10:15.440088 1343 raft_consensus.cc:729] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bca3dcbe084701be54e2a3da7a54f9, State: Initialized, Role: FOLLOWER
I20260812 21:10:15.440233 1346 ts_tablet_manager.cc:1428] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 21:10:15.440383 797 heartbeater.cc:503] Master 127.11.207.190:35967 was elected leader, sending a full tablet report...
I20260812 21:10:15.440552 1343 consensus_queue.cc:260] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.440995 1343 ts_tablet_manager.cc:1428] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260812 21:10:15.441195 1049 heartbeater.cc:503] Master 127.11.207.190:35967 was elected leader, sending a full tablet report...
I20260812 21:10:15.441823 1350 tablet_bootstrap.cc:492] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: No bootstrap required, opened a new log
I20260812 21:10:15.441952 1350 ts_tablet_manager.cc:1397] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260812 21:10:15.443936 1350 raft_consensus.cc:348] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.444067 1350 raft_consensus.cc:374] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:10:15.444097 1350 raft_consensus.cc:729] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba4fb05ecb34709b67017458fb3592e, State: Initialized, Role: FOLLOWER
I20260812 21:10:15.445468 1350 consensus_queue.cc:260] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.445816 1350 ts_tablet_manager.cc:1428] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 21:10:15.445875 1308 heartbeater.cc:503] Master 127.11.207.190:35967 was elected leader, sending a full tablet report...
Aug 12 21:10:15 dist-test-slave-lkx2 krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
W20260812 21:10:15.579887 798 tablet.cc:2369] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:10:15.581768 1356 raft_consensus.cc:482] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:10:15.581900 1356 raft_consensus.cc:504] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.582347 1356 leader_election.cc:288] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a78ecedf7f44e7984d974be4369eb0 (127.11.207.129:40507), 4ba4fb05ecb34709b67017458fb3592e (127.11.207.131:38499)
I20260812 21:10:15.590775 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.582574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:38499 (local address 127.11.207.130:42967)
0812 21:10:15.582647 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.582669 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.582720 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.582884 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.582885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.582886 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:10:15.582964 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.582966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.584023 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.584025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.585281 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.585285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.590429 (+ 5144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.590430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.590581 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.590585 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.590636 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.590690 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 21:10:15.591071 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.582695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:42967 (local address 127.11.207.131:38499)
0812 21:10:15.582758 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.582760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.582770 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:10:15.582801 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.582802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.582811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.582870 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:10:15.583049 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.583813 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.589723 (+ 5910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.590307 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.590383 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.590392 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.590955 (+ 563us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.591006 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:10:15.591421 1216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b162e045a01b4bfe900bfd798284d0a0" candidate_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba4fb05ecb34709b67017458fb3592e" is_pre_election: true
I20260812 21:10:15.591593 1216 raft_consensus.cc:2454] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4bca3dcbe084701be54e2a3da7a54f9 in term 0.
I20260812 21:10:15.591851 845 leader_election.cc:302] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ba4fb05ecb34709b67017458fb3592e, f4bca3dcbe084701be54e2a3da7a54f9; no voters:
I20260812 21:10:15.591884 1337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.583485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:47785 (local address 127.11.207.129:40507)
0812 21:10:15.583531 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.583533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.587552 (+ 4019us) server_negotiation.cc:407] Connection header received
0812 21:10:15.587647 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.587648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.587658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.587712 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:10:15.587933 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.588750 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.590303 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.590950 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.591006 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.591014 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.591766 (+ 752us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.591814 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:10:15.591990 1356 raft_consensus.cc:2781] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:10:15.592031 1356 raft_consensus.cc:482] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:10:15.592051 1356 raft_consensus.cc:3037] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:10:15.592365 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.583159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40507 (local address 127.11.207.130:47785)
0812 21:10:15.587477 (+ 4318us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.587491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:10:15.587590 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:10:15.587761 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:10:15.587762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:10:15.587764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:10:15.587860 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.587862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.588869 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.588870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.590196 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:10:15.590200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.591437 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.591439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:10:15.591512 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.591601 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 21:10:15.592195 (+ 594us) client_negotiation.cc:240] Negotiation successful
0812 21:10:15.592241 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4299,"thread_start_us":141,"threads_started":1}
I20260812 21:10:15.593091 1356 raft_consensus.cc:504] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.593243 1356 leader_election.cc:288] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [CANDIDATE]: Term 1 election: Requested vote from peers e9a78ecedf7f44e7984d974be4369eb0 (127.11.207.129:40507), 4ba4fb05ecb34709b67017458fb3592e (127.11.207.131:38499)
I20260812 21:10:15.593429 700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b162e045a01b4bfe900bfd798284d0a0" candidate_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a78ecedf7f44e7984d974be4369eb0" is_pre_election: true
I20260812 21:10:15.593616 700 raft_consensus.cc:2454] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4bca3dcbe084701be54e2a3da7a54f9 in term 0.
I20260812 21:10:15.593955 700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b162e045a01b4bfe900bfd798284d0a0" candidate_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a78ecedf7f44e7984d974be4369eb0"
I20260812 21:10:15.594025 700 raft_consensus.cc:3037] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:10:15.595108 700 raft_consensus.cc:2454] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4bca3dcbe084701be54e2a3da7a54f9 in term 1.
I20260812 21:10:15.595275 846 leader_election.cc:302] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e9a78ecedf7f44e7984d974be4369eb0, f4bca3dcbe084701be54e2a3da7a54f9; no voters:
I20260812 21:10:15.595323 1216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b162e045a01b4bfe900bfd798284d0a0" candidate_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba4fb05ecb34709b67017458fb3592e"
I20260812 21:10:15.595414 1216 raft_consensus.cc:3037] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:10:15.596414 1216 raft_consensus.cc:2454] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4bca3dcbe084701be54e2a3da7a54f9 in term 1.
I20260812 21:10:15.596423 1356 raft_consensus.cc:2781] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:10:15.596698 1356 raft_consensus.cc:686] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 1 LEADER]: Becoming Leader. State: Replica: f4bca3dcbe084701be54e2a3da7a54f9, State: Running, Role: LEADER
I20260812 21:10:15.596809 1356 consensus_queue.cc:237] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } }
I20260812 21:10:15.598549 353 catalog_manager.cc:5495] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4bca3dcbe084701be54e2a3da7a54f9 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4bca3dcbe084701be54e2a3da7a54f9" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 } health_report { overall_health: UNKNOWN } } }
I20260812 21:10:15.609810 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.606074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.11.207.130:38349)
0812 21:10:15.606472 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 21:10:15.606474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:10:15.606483 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:10:15.606523 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:10:15.606524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:10:15.606545 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:10:15.606615 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:10:15.606835 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.607576 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.608719 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:10:15.609027 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:10:15.609084 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:10:15.609278 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:10:15.609415 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:10:15.609484 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:10:15.609708 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 21:10:15.609729 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371}
I20260812 21:10:15.612617 1216 raft_consensus.cc:1261] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 1 FOLLOWER]: Refusing update from remote peer f4bca3dcbe084701be54e2a3da7a54f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:10:15.613061 1356 consensus_queue.cc:1035] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba4fb05ecb34709b67017458fb3592e" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 21:10:15.613282 1050 tablet.cc:2369] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:10:15.615463 700 raft_consensus.cc:1261] T b162e045a01b4bfe900bfd798284d0a0 P e9a78ecedf7f44e7984d974be4369eb0 [term 1 FOLLOWER]: Refusing update from remote peer f4bca3dcbe084701be54e2a3da7a54f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:10:15.615816 1356 consensus_queue.cc:1035] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a78ecedf7f44e7984d974be4369eb0" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:10:15.620613 1365 mvcc.cc:205] Tried to move back new op lower bound from 7317786687946137600 to 7317786687885225984. Current Snapshot: MvccSnapshot[applied={T|T < 7317786687946137600}]
I20260812 21:10:15.621240 1366 mvcc.cc:205] Tried to move back new op lower bound from 7317786687946137600 to 7317786687885225984. Current Snapshot: MvccSnapshot[applied={T|T < 7317786687946137600}]
I20260812 21:10:15.623973 353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:10:15.624069 353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:10:15.624984 353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ccc7c2ab121479babcc224a8509fa9c: Sending DeleteTablet for 3 replicas of tablet b162e045a01b4bfe900bfd798284d0a0
I20260812 21:10:15.625445 928 tablet_service.cc:1514] Processing DeleteTablet for tablet b162e045a01b4bfe900bfd798284d0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:10:15 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:36678
I20260812 21:10:15.625799 1177 tablet_service.cc:1514] Processing DeleteTablet for tablet b162e045a01b4bfe900bfd798284d0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:10:15 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:59050
I20260812 21:10:15.626876 662 tablet_service.cc:1514] Processing DeleteTablet for tablet b162e045a01b4bfe900bfd798284d0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:10:15 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:60122
I20260812 21:10:15.628496 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 540
I20260812 21:10:15.649277 1378 tablet_replica.cc:330] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: stopping tablet replica
I20260812 21:10:15.649510 1380 tablet_replica.cc:330] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: stopping tablet replica
I20260812 21:10:15.656184 1380 raft_consensus.cc:2229] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:10:15.656476 1378 raft_consensus.cc:2229] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:10:15.656637 1380 raft_consensus.cc:2258] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:10:15.656854 1378 raft_consensus.cc:2258] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:10:15.657591 1378 ts_tablet_manager.cc:1905] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 21:10:15.660701 332 connection.cc:535] server connection from 127.11.207.129:38153 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:10:15.661231 1380 ts_tablet_manager.cc:1905] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:10:15.662853 1380 ts_tablet_manager.cc:1918] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:10:15.662933 1380 log.cc:1196] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/b162e045a01b4bfe900bfd798284d0a0
I20260812 21:10:15.663501 1380 ts_tablet_manager.cc:1939] T b162e045a01b4bfe900bfd798284d0a0 P 4ba4fb05ecb34709b67017458fb3592e: Deleting consensus metadata
I20260812 21:10:15.664016 334 catalog_manager.cc:4841] TS 4ba4fb05ecb34709b67017458fb3592e (127.11.207.131:38499): tablet b162e045a01b4bfe900bfd798284d0a0 (table test-table [id=e1d373f96c5546ba98331bc25f80b47a]) successfully deleted
W20260812 21:10:15.664036 335 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:40507 (error 108)
I20260812 21:10:15.664755 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.664610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40507 (local address 127.0.0.1:45156)
0812 21:10:15.664664 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.664720 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 21:10:15.664960 335 catalog_manager.cc:4584] TS e9a78ecedf7f44e7984d974be4369eb0 (127.11.207.129:40507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b162e045a01b4bfe900bfd798284d0a0: Network error: Client connection negotiation failed: client connection to 127.11.207.129:40507: connect: Connection refused (error 111)
I20260812 21:10:15.668514 1378 ts_tablet_manager.cc:1918] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:10:15.668696 1378 log.cc:1196] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/b162e045a01b4bfe900bfd798284d0a0
I20260812 21:10:15.669035 1378 ts_tablet_manager.cc:1939] T b162e045a01b4bfe900bfd798284d0a0 P f4bca3dcbe084701be54e2a3da7a54f9: Deleting consensus metadata
I20260812 21:10:15.669556 334 catalog_manager.cc:4841] TS f4bca3dcbe084701be54e2a3da7a54f9 (127.11.207.130:38349): tablet b162e045a01b4bfe900bfd798284d0a0 (table test-table [id=e1d373f96c5546ba98331bc25f80b47a]) successfully deleted
I20260812 21:10:15.672498 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 808
I20260812 21:10:15.696475 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 1058
I20260812 21:10:15.718986 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.718854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40507 (local address 127.0.0.1:45162)
0812 21:10:15.718908 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.718938 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:10:15.719386 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:10:15.719328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:40507 (local address 127.0.0.1:45178)
0812 21:10:15.719354 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0812 21:10:15.719369 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 21:10:15.725219 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 302
I20260812 21:10:15.743009 12094 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 21:10:16.089367 12094 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 21:10:16.089480 12094 mini_ranger.cc:66] Stopping Ranger...
I20260812 21:10:16.444453 12094 mini_ranger.cc:68] Stopped Ranger
2026-08-12 21:10:16.444 UTC [31934] LOG: received smart shutdown request
2026-08-12 21:10:16.448 UTC [31934] LOG: background worker "logical replication launcher" (PID 31950) exited with exit code 1
2026-08-12 21:10:16.448 UTC [31939] LOG: shutting down
2026-08-12 21:10:16.618 UTC [31934] LOG: database system is shut down
2026-08-12T21:10:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50212 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:10:16 dist-test-slave-lkx2 krb5kdc[1397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:10:16 dist-test-slave-lkx2 krb5kdc[1397](info): set up 2 sockets
Aug 12 21:10:16 dist-test-slave-lkx2 krb5kdc[1397](info): commencing operation
krb5kdc: starting...
W20260812 21:10:17.914901 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:10:17 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:10:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:10:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/rangeradmin_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/rangeradmin_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/rangerlookup_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/rangerlookup_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
I20260812 21:10:18.087811 12094 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwFLJxB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/postgres -l logfile start
2026-08-12 21:10:20.501 UTC [1454] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 21:10:20.501 UTC [1454] LOG: listening on IPv4 address "127.11.207.148", port 53635
2026-08-12 21:10:20.504 UTC [1454] LOG: listening on Unix socket "/tmp/.s.PGSQL.53635"
2026-08-12 21:10:20.525 UTC [1458] LOG: database system was shut down at 2026-08-12 21:10:19 UTC
2026-08-12 21:10:20.528 UTC [1454] LOG: database system is ready to accept connections
I20260812 21:10:21.645294 12094 mini_postgres.cc:96] Postgres bound to 53635
2026-08-12 21:10:21.652 UTC [1472] FATAL: database "slave" does not exist
127.11.207.148:53635 - accepting connections
I20260812 21:10:21.652619 12094 mini_ranger.cc:161] Starting Ranger...
I20260812 21:10:21.667343 12094 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 21:10:22.122437 12094 mini_ranger.cc:87] Created ranger Postgres database
I20260812 21:10:22.122516 12094 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-admin
2026-08-12 21:10:22,402 [I] DB FLAVOR :POSTGRES
2026-08-12 21:10:22,403 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:10:22,403 [I] Checking connection..
2026-08-12 21:10:22,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 21:10:22,666 [I] Checking connection passed.
2026-08-12 21:10:22,667 [I] --------- Verifying version history table ---------
2026-08-12 21:10:22,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:10:22,917 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:10:22,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:10:23,171 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:10:23,171 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 21:10:23,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 21:10:23.407 UTC [1610] WARNING: there is no transaction in progress
2026-08-12 21:10:23,416 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 21:10:23,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:10:23,679 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 21:10:23,679 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:10:23,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 21:10:23,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 21:10:24,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lkx2.c.gcp-upstream.internal','N') ;"
2026-08-12 21:10:24,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:10:24,723 [I] Table x_portal_user does not exist in database ranger
2026-08-12 21:10:24,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:10:24,972 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 21:10:24,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:10:25,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:10:25,507 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 21:10:25,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 21:10:26.483 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.499 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.515 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.531 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.547 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.565 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.625 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.632 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.647 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.657 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.668 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.681 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.692 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.699 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.706 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:26.713 UTC [1815] WARNING: there is no transaction in progress
2026-08-12 21:10:27,351 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 21:10:27,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 21:10:27,606 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 21:10:27,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:10:27,871 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 21:10:27,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:10:28,123 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 21:10:28,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:10:28,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:10:28,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lkx2.c.gcp-upstream.internal';"
2026-08-12 21:10:28,906 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 21:10:28,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 21:10:29,175 [I] DB_PATCHES have already been applied
I20260812 21:10:29.181545 12094 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwFLJxB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 21:10:29.181612 12094 mini_ranger.cc:193] Using host: 127.11.207.148
I20260812 21:10:29.185755 12094 mini_ranger.cc:227] Ranger admin URL: http://127.11.207.148:58773
Aug 12, 2026 9:10:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:10:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:10:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 9:10:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 9:10:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 9:10:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 9:10:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 21:10:30.508690 12094 mini_ranger.cc:160] Time spent starting Ranger: real 8.856s user 0.001s sys 0.010s
Aug 12, 2026 9:10:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 9:10:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 9:10:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 9:10:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 9:10:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 21:10:53.012488 12094 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.207.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.207.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/rangerkms_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.207.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/rangerkms_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.207.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.207.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.207.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/HTTP_127.11.207.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 21:10:53 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569053, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 21:10:53.102293 12094 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 21:10:53.113163 12094 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 21:10:53.939663 12094 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 21:10:53.939739 12094 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms
I20260812 21:10:53.939757 12094 mini_ranger_kms.cc:227] Using postgres at 127.11.207.148:53635
2026-08-12 21:10:54,108 [I] DB FLAVOR :POSTGRES
2026-08-12 21:10:54,108 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:10:54,108 [I] Checking connection
2026-08-12 21:10:54,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 21:10:54,378 [I] connection success
2026-08-12 21:10:54,378 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 21:10:54,378 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 21:10:54,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 21:10:54,646 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 21:10:54,647 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:10:54,647 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 21:10:54,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.207.148:53635/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 21:10:54,907 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 21:10:55.429160 12094 mini_ranger_kms.cc:310] Created kms service
I20260812 21:10:55.635272 12094 mini_ranger_kms.cc:326] Created kudu user
I20260812 21:10:55.685406 12094 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 21:10:55 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569053, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:10:56.098829 12094 mini_ranger_kms.cc:384] Added ranger policy
I20260812 21:10:56.099040 12094 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwFLJxB/build/release/bin/postgresql.jar:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwFLJxB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 21:10:56.099062 12094 mini_ranger_kms.cc:240] Using host: 127.11.207.148
I20260812 21:10:56.103857 12094 mini_ranger_kms.cc:276] Ranger KMS PID: 2134
I20260812 21:10:56.103935 12094 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.207.148:56913
Aug 12, 2026 9:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 9:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 9:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 9:10:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 21:10:58 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786569058, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.207.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:10:58.699211 12094 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.597s user 0.009s sys 0.144s
I20260812 21:10:58.699306 12094 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 21:10:58.699354 12094 mini_ranger_kms.cc:401] 127.11.207.148:56913/kms/v1/keys
Aug 12 21:10:58 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569058, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.207.148@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
Aug 12 21:10:59 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569058, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.207.148@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
Aug 12 21:11:01 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:11:01.694720 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37783
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:37783
--ranger_config_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:01.795186 2240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:01.795389 2240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:01.795410 2240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:01.796770 2240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:01.796818 2240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:01.796831 2240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:01.796842 2240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:01.796854 2240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:01.798498 2240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.207.148:56913/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:37783
--ranger_config_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:01.798748 2240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:01.798959 2240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:01.804139 2249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:01.804162 2251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:01.804211 2240 server_base.cc:1034] running on GCE node
W20260812 21:11:01.804139 2253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:01.804625 2240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:01.804919 2240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:01.806054 2240 hybrid_clock.cc:648] HybridClock initialized: now 1786569061806036 us; error 37 us; skew 500 ppm
Aug 12 21:11:01 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:01.808588 2240 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:01.809041 2240 webserver.cc:460] Webserver started at http://127.11.207.190:46867/ using document root <none> and password file <none>
I20260812 21:11:01.809258 2240 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:01.809301 2240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:01.809432 2240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:11:01 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569061, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:01.858543 2240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "d7e495d7882741e68d3d26097ade29a2"
format_stamp: "Formatted at 2026-08-12 21:11:01 on dist-test-slave-lkx2"
server_key: "761f44eef659276144c3b3d0ad5a6eee"
server_key_iv: "9926e8e5f4f8b1ed1ae521f74c007692"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:01.858942 2240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "d7e495d7882741e68d3d26097ade29a2"
format_stamp: "Formatted at 2026-08-12 21:11:01 on dist-test-slave-lkx2"
server_key: "761f44eef659276144c3b3d0ad5a6eee"
server_key_iv: "9926e8e5f4f8b1ed1ae521f74c007692"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:01.860317 2240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:11:01.872100 2271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:01.872299 2240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:01.872447 2240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "d7e495d7882741e68d3d26097ade29a2"
format_stamp: "Formatted at 2026-08-12 21:11:01 on dist-test-slave-lkx2"
server_key: "761f44eef659276144c3b3d0ad5a6eee"
server_key_iv: "9926e8e5f4f8b1ed1ae521f74c007692"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:01.872537 2240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:01.881232 2240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:01.883282 2240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:01.883428 2240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:01.888085 2240 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:37783
I20260812 21:11:01.888113 2369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:37783 every 8 connection(s)
I20260812 21:11:01.888607 2240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:01.889328 2370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:01.891709 2370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Bootstrap starting.
I20260812 21:11:01.892127 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 2240
I20260812 21:11:01.892230 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:11:01.892431 12094 external_mini_cluster.cc:1428] Setting key 5c356ec4dc730d4b6ee999fa877044c4
I20260812 21:11:01.892491 2370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:01.892835 2370 log.cc:824] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:01.893621 2370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: No bootstrap required, opened a new log
I20260812 21:11:01.895918 2370 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:01.896101 2370 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:01.896132 2370 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e495d7882741e68d3d26097ade29a2, State: Initialized, Role: FOLLOWER
I20260812 21:11:01.896260 2370 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:01.896324 2370 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:01.896374 2370 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:01.896412 2370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:01.897029 2370 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:01.897145 2370 leader_election.cc:302] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e495d7882741e68d3d26097ade29a2; no voters:
I20260812 21:11:01.897341 2370 leader_election.cc:288] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:11:01.897396 2382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:01.897569 2382 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 LEADER]: Becoming Leader. State: Replica: d7e495d7882741e68d3d26097ade29a2, State: Running, Role: LEADER
I20260812 21:11:01.897687 2370 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:01.897697 2382 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:01.898028 2382 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7e495d7882741e68d3d26097ade29a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } } }
I20260812 21:11:01.898088 2382 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:01.898200 2382 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e495d7882741e68d3d26097ade29a2. Latest consensus state: current_term: 1 leader_uuid: "d7e495d7882741e68d3d26097ade29a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } } }
I20260812 21:11:01.898258 2382 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:01.898528 2391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:01.899052 2391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:01.899395 2370 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 12 21:11:01 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:01.908417 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:01.892992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33354 (local address 127.11.207.190:37783)
0812 21:11:01.893327 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:01.893337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:01.893371 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:11:01.893941 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:01.893950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:01.894022 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:01.894176 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:01.895066 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:01.896519 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:01.897708 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:01.898373 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:01.898473 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:01.902350 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:01.902412 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:01.902412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:01.902527 (+ 115us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:01.905957 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:01.906775 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:01.906779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:01.906779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:01.906892 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:01.907139 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:01.907140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:01.907140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:01.907495 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:01.907653 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:01.907953 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 21:11:01.908129 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":79,"threads_started":1}
I20260812 21:11:01.912988 2391 catalog_manager.cc:1347] Generated new cluster ID: 19b03138a6584f49948323f5109d5233
I20260812 21:11:01.913049 2391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:11:01.921682 2391 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:11:01.922539 2391 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:01.930809 2391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Generated new TSK 0
I20260812 21:11:01.931105 2391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:11:02.635556 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37783
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:02.746099 2454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:02.746299 2454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:02.746321 2454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:02.747460 2454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:02.747502 2454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:02.747531 2454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:02.749048 2454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.207.148:56913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:02.749292 2454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:02.749532 2454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:02.754565 2465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:02.754565 2463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:02.754843 2462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:02.755082 2454 server_base.cc:1034] running on GCE node
I20260812 21:11:02.755275 2454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:02.755499 2454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:02.756656 2454 hybrid_clock.cc:648] HybridClock initialized: now 1786569062756639 us; error 38 us; skew 500 ppm
Aug 12 21:11:02 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:02.759308 2454 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:02.759719 2454 webserver.cc:460] Webserver started at http://127.11.207.129:39015/ using document root <none> and password file <none>
I20260812 21:11:02.759967 2454 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:02.760020 2454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:02.760138 2454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:11:02 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:02.774786 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "e8e3e5120d4644ada84fc7ea8572a2f9"
format_stamp: "Formatted at 2026-08-12 21:11:02 on dist-test-slave-lkx2"
server_key: "ee774cf21fb45e05d3d8ee4c4ca62b22"
server_key_iv: "cb6c3fc195d472d67f4481a9ab0cc4c3"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:02.775240 2454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "e8e3e5120d4644ada84fc7ea8572a2f9"
format_stamp: "Formatted at 2026-08-12 21:11:02 on dist-test-slave-lkx2"
server_key: "ee774cf21fb45e05d3d8ee4c4ca62b22"
server_key_iv: "cb6c3fc195d472d67f4481a9ab0cc4c3"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:02.778102 2454 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 21:11:02.801586 2475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:02.801806 2454 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260812 21:11:02.801913 2454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "e8e3e5120d4644ada84fc7ea8572a2f9"
format_stamp: "Formatted at 2026-08-12 21:11:02 on dist-test-slave-lkx2"
server_key: "ee774cf21fb45e05d3d8ee4c4ca62b22"
server_key_iv: "cb6c3fc195d472d67f4481a9ab0cc4c3"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:02.802024 2454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:02.820406 2454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:02.822877 2454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:02.823104 2454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:02.823437 2454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:02.823890 2454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:02.823978 2454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:02.824071 2454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:02.824146 2454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:02.830912 2454 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:34511
I20260812 21:11:02.831449 2454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:11:02.833293 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 2454
I20260812 21:11:02.833422 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:11:02.833614 12094 external_mini_cluster.cc:1428] Setting key c45d66d8359e742ff9f2c466668c0108
I20260812 21:11:02.833947 2705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:34511 every 8 connection(s)
Aug 12 21:11:02 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569062, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
I20260812 21:11:02.846451 2711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:02.834683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37783 (local address 127.11.207.129:40011)
0812 21:11:02.835299 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:02.835347 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:02.835994 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:02.836446 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:02.836451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:02.836711 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:02.836994 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:02.837000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:02.838034 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:02.838035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:02.839366 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:02.839370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:02.839935 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:02.839936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:02.840102 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:02.840783 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:02.840794 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:02.842521 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:02.844738 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:02.844746 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:02.844749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:02.845050 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:02.845279 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:02.845280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:02.845281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:02.845435 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:02.845731 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:02.845738 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:02.845923 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 21:11:02.846019 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:11:02.846185 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
I20260812 21:11:02.847078 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:02.834988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:40011 (local address 127.11.207.190:37783)
0812 21:11:02.836186 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:02.836188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:02.836196 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:11:02.836220 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:02.836221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:02.836233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:02.836293 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:02.837111 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:02.837906 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:02.839491 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:02.839805 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:02.839873 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:02.842633 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:02.842634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:02.842635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:02.842644 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:02.844636 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:02.845139 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:02.845140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:02.845140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:02.845207 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:02.845512 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:02.845513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:02.845513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:02.845661 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:02.846538 (+ 877us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:02.846585 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 21:11:02.846693 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1165,"thread_start_us":82,"threads_started":1}
I20260812 21:11:02.847958 2707 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37783
I20260812 21:11:02.848078 2707 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:02.848280 2707 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:02.848963 2302 ts_manager.cc:194] Registered new tserver with Master: e8e3e5120d4644ada84fc7ea8572a2f9 (127.11.207.129:34511)
I20260812 21:11:02.849865 2302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:40011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:11:02.866806 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37783
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:02.987823 2719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:02.988125 2719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:02.988168 2719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:02.989872 2719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:02.989934 2719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:02.989974 2719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:02.992096 2719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.207.148:56913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:02.992556 2719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:02.992846 2719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:02.999575 2734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:02.999897 2737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.000159 2733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:03.001214 2719 server_base.cc:1034] running on GCE node
I20260812 21:11:03.001422 2719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:03.001708 2719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:03.004994 2719 hybrid_clock.cc:648] HybridClock initialized: now 1786569063004959 us; error 46 us; skew 500 ppm
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:03.008500 2719 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:03.009111 2719 webserver.cc:460] Webserver started at http://127.11.207.130:39825/ using document root <none> and password file <none>
I20260812 21:11:03.009423 2719 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:03.009498 2719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:03.009704 2719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:03.025722 2719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "c9552a3bbee64587be7f9d3ec9dabe9b"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "bbc0dcb57b0acd3083c96ea0d4fa07dc"
server_key_iv: "1f70d4d11d7a8802fdf52c1707a50bef"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.026208 2719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "c9552a3bbee64587be7f9d3ec9dabe9b"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "bbc0dcb57b0acd3083c96ea0d4fa07dc"
server_key_iv: "1f70d4d11d7a8802fdf52c1707a50bef"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.036912 2719 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.003s sys 0.000s
I20260812 21:11:03.068501 2758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.068753 2719 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260812 21:11:03.068888 2719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "c9552a3bbee64587be7f9d3ec9dabe9b"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "bbc0dcb57b0acd3083c96ea0d4fa07dc"
server_key_iv: "1f70d4d11d7a8802fdf52c1707a50bef"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.069010 2719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:03.090014 2719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:03.092710 2719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:03.092962 2719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:03.093253 2719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:03.093640 2719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:03.093680 2719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.093711 2719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:03.093763 2719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.100982 2719 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:34287
I20260812 21:11:03.101363 2719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:03.101958 2978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:34287 every 8 connection(s)
I20260812 21:11:03.105324 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 2719
I20260812 21:11:03.105435 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:11:03.105623 12094 external_mini_cluster.cc:1428] Setting key 91eaf69f5120e71aa9e3448afed02df6
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:03.113941 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:03.102118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37783 (local address 127.11.207.130:39881)
0812 21:11:03.102348 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:03.102390 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:03.102963 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:03.103356 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:03.103361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:03.103641 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:03.103929 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:03.103937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.105041 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.105042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:03.106954 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:03.106958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.107564 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.107565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:03.107732 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:03.108405 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:03.108415 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:03.110089 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:03.112181 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:03.112188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:03.112191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:03.112526 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:03.112739 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:03.112740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:03.112741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:03.112893 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:03.113197 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:03.113204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:03.113398 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 21:11:03.113510 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 21:11:03.113674 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
I20260812 21:11:03.114676 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:03.102233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:39881 (local address 127.11.207.190:37783)
0812 21:11:03.102283 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:03.102285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:03.102484 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:11:03.103098 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:03.103100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:03.103117 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:03.103196 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:03.104063 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.104931 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.107058 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.107393 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.107439 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:03.110206 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:03.110208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:03.110208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:03.110217 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:03.112081 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:03.112613 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:03.112614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:03.112614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:03.112677 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:03.112964 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:03.112965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:03.112965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:03.113114 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:03.113171 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:03.114463 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0812 21:11:03.114562 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:11:03.115105 2979 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37783
I20260812 21:11:03.115222 2979 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:03.115429 2979 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:03.115859 2302 ts_manager.cc:194] Registered new tserver with Master: c9552a3bbee64587be7f9d3ec9dabe9b (127.11.207.130:34287)
I20260812 21:11:03.116403 2302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:39881
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:11:03.143743 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37783
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:03.268692 2992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:03.268999 2992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:03.269044 2992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:03.270814 2992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:03.270884 2992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:03.270926 2992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:03.273413 2992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.207.148:56913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:03.273829 2992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:03.274129 2992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:03.280982 3007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.281078 3004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.280982 3010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:03.282178 2992 server_base.cc:1034] running on GCE node
I20260812 21:11:03.282387 2992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:03.282637 2992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:03.283797 2992 hybrid_clock.cc:648] HybridClock initialized: now 1786569063283777 us; error 35 us; skew 500 ppm
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:03.286942 2992 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:03.287442 2992 webserver.cc:460] Webserver started at http://127.11.207.131:41231/ using document root <none> and password file <none>
I20260812 21:11:03.287663 2992 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:03.287698 2992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:03.287786 2992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:03.301357 2992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "956493b71299477980bb27f6f0b004f9"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "759437f2fd45d72843218b3874efd0f7"
server_key_iv: "fe7665ccfaa8d1decdd9fe99c4595391"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.301936 2992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "956493b71299477980bb27f6f0b004f9"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "759437f2fd45d72843218b3874efd0f7"
server_key_iv: "fe7665ccfaa8d1decdd9fe99c4595391"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.303885 2992 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:03.318197 3024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.318447 2992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:11:03.318562 2992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "956493b71299477980bb27f6f0b004f9"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "759437f2fd45d72843218b3874efd0f7"
server_key_iv: "fe7665ccfaa8d1decdd9fe99c4595391"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.318681 2992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:03.351179 2992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:03.354023 2992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:03.354256 2992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:03.354586 2992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:03.355036 2992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:03.355134 2992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.355229 2992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:03.355289 2992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.362103 2992 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:38693
I20260812 21:11:03.362706 2992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:11:03.372596 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 2992
I20260812 21:11:03.372704 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:11:03.372908 12094 external_mini_cluster.cc:1428] Setting key 5fbe1dd8d76ffd02690ba1125ec5fadd
I20260812 21:11:03.372982 3207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:38693 every 8 connection(s)
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:03.397955 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:03.381396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:54203 (local address 127.11.207.190:37783)
0812 21:11:03.381444 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:03.381446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:03.381651 (+ 205us) server_negotiation.cc:407] Connection header received
0812 21:11:03.382417 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:03.382419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:03.382432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:03.382465 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:03.383372 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.384092 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.388260 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.388583 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.388637 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:03.392851 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:03.392852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:03.392853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:03.392862 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:03.394614 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:03.396779 (+ 2165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:03.396781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:03.396781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:03.396851 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:03.397146 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:03.397147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:03.397148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:03.397291 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:03.397363 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:03.397764 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 21:11:03.397856 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:11:03.398773 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:03.381053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37783 (local address 127.11.207.131:54203)
0812 21:11:03.381522 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:03.381571 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:03.382273 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:03.382671 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:03.382675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:03.382951 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:03.383243 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:03.383250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.384217 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.384218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:03.388149 (+ 3931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:03.388153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.388750 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.388751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:03.388896 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:03.389527 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:03.389538 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:03.392723 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:03.396016 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:03.396026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:03.396029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:03.396682 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:03.396925 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:03.396926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:03.396926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:03.397072 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:03.397402 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:03.397409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:03.397594 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 21:11:03.398331 (+ 737us) client_negotiation.cc:240] Negotiation successful
0812 21:11:03.398519 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":125,"threads_started":1}
I20260812 21:11:03.399585 3208 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37783
I20260812 21:11:03.399685 3208 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:03.399881 3208 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:03.400324 2302 ts_manager.cc:194] Registered new tserver with Master: 956493b71299477980bb27f6f0b004f9 (127.11.207.131:38693)
I20260812 21:11:03.400854 2302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:54203
I20260812 21:11:03.407392 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:11:03.408097 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 2454
I20260812 21:11:03.444880 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 2719
I20260812 21:11:03.484647 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 2992
I20260812 21:11:03.495007 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 2240
I20260812 21:11:03.513731 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37783
--webserver_interface=127.11.207.190
--webserver_port=46867
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:37783
--ranger_config_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:03.594671 3213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:03.594897 3213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:03.594918 3213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:03.596017 3213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:03.596050 3213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:03.596063 3213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:03.596074 3213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:03.596086 3213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:03.597667 3213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:37783
--ranger_config_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=46867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:03.597873 3213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:03.598047 3213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:03.602665 3225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.602674 3222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.602677 3221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:03.603051 3213 server_base.cc:1034] running on GCE node
I20260812 21:11:03.603251 3213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:03.603498 3213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:03.604638 3213 hybrid_clock.cc:648] HybridClock initialized: now 1786569063604622 us; error 33 us; skew 500 ppm
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:03.607007 3213 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:03.607452 3213 webserver.cc:460] Webserver started at http://127.11.207.190:46867/ using document root <none> and password file <none>
I20260812 21:11:03.607671 3213 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:03.607728 3213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:03.648726 3213 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:03.649583 3242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.649832 3213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:03.649955 3213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "d7e495d7882741e68d3d26097ade29a2"
format_stamp: "Formatted at 2026-08-12 21:11:01 on dist-test-slave-lkx2"
server_key: "761f44eef659276144c3b3d0ad5a6eee"
server_key_iv: "9926e8e5f4f8b1ed1ae521f74c007692"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.650363 3213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:03.661736 3213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:03.663599 3213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:03.663744 3213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:03.667742 3213 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:37783
I20260812 21:11:03.667801 3342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:37783 every 8 connection(s)
I20260812 21:11:03.668143 3213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:03.669390 3343 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:11:03.669790 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 3213
I20260812 21:11:03.670027 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:34511
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:37783
--webserver_port=39015
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:03.670104 3343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Bootstrap starting.
I20260812 21:11:03.677973 3343 log.cc:824] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:03.679618 3343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:03.679864 3343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Bootstrap complete.
I20260812 21:11:03.682122 3343 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:03.682363 3343 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e495d7882741e68d3d26097ade29a2, State: Initialized, Role: FOLLOWER
I20260812 21:11:03.682520 3343 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:03.682595 3343 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:03.682646 3343 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:03.682691 3343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:11:03.683655 3343 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:03.683749 3343 leader_election.cc:302] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e495d7882741e68d3d26097ade29a2; no voters:
I20260812 21:11:03.683907 3343 leader_election.cc:288] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:11:03.683955 3348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:11:03.684088 3348 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [term 2 LEADER]: Becoming Leader. State: Replica: d7e495d7882741e68d3d26097ade29a2, State: Running, Role: LEADER
I20260812 21:11:03.684180 3343 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:03.684178 3348 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } }
I20260812 21:11:03.684507 3350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d7e495d7882741e68d3d26097ade29a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } } }
I20260812 21:11:03.684615 3350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:03.684509 3348 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e495d7882741e68d3d26097ade29a2. Latest consensus state: current_term: 2 leader_uuid: "d7e495d7882741e68d3d26097ade29a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e495d7882741e68d3d26097ade29a2" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37783 } } }
I20260812 21:11:03.684823 3348 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:03.685283 3363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:03.685629 3343 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 21:11:03.685761 3363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:03.685892 3363 catalog_manager.cc:1259] Loaded cluster ID: 19b03138a6584f49948323f5109d5233
I20260812 21:11:03.685925 3363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:11:03.690269 3363 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:03.690666 3363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Loaded TSK: 0
I20260812 21:11:03.690900 3363 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:11:03.757817 3345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:03.758009 3345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:03.758040 3345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:03.759264 3345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:03.759379 3345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:03.759470 3345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:03.761607 3345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:34511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=39015
--tserver_master_addrs=127.11.207.190:37783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:03.762086 3345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:03.762424 3345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:03.767976 3412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.768011 3417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.767969 3411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:03.768774 3345 server_base.cc:1034] running on GCE node
I20260812 21:11:03.768981 3345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:03.769217 3345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:03.770398 3345 hybrid_clock.cc:648] HybridClock initialized: now 1786569063770372 us; error 39 us; skew 500 ppm
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:03.773102 3345 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:03.774098 3345 webserver.cc:460] Webserver started at http://127.11.207.129:39015/ using document root <none> and password file <none>
I20260812 21:11:03.774312 3345 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:03.774364 3345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:03.811981 3345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:03.817011 3430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.817325 3345 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260812 21:11:03.817447 3345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "e8e3e5120d4644ada84fc7ea8572a2f9"
format_stamp: "Formatted at 2026-08-12 21:11:02 on dist-test-slave-lkx2"
server_key: "ee774cf21fb45e05d3d8ee4c4ca62b22"
server_key_iv: "cb6c3fc195d472d67f4481a9ab0cc4c3"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:03.818015 3345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:03.831813 3345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:03.836606 3345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:03.836800 3345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:03.837103 3345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:03.837505 3345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:03.837646 3345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.837697 3345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:03.837718 3345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:03.844991 3345 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:34511
I20260812 21:11:03.845171 3630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:34511 every 8 connection(s)
I20260812 21:11:03.845554 3345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:11:03.851087 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 3345
I20260812 21:11:03.851400 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:34287
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:37783
--webserver_port=39825
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:03.878013 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:03.846447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:57331 (local address 127.11.207.190:37783)
0812 21:11:03.856560 (+ 10113us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:03.856569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:03.856607 (+ 38us) server_negotiation.cc:407] Connection header received
0812 21:11:03.856763 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:03.856769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:03.856854 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:03.857005 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:03.857818 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.859288 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.860803 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.863249 (+ 2446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.863843 (+ 594us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:03.867369 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:03.867374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:03.867374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:03.867389 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:03.873978 (+ 6589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:03.876226 (+ 2248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:03.876230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:03.876231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:03.876398 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:03.876762 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:03.876764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:03.876764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:03.877126 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:03.877224 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:03.877642 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 21:11:03.877785 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10054,"thread_start_us":85,"threads_started":1}
I20260812 21:11:03.878054 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:03.846344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37783 (local address 127.11.207.129:57331)
0812 21:11:03.855630 (+ 9286us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:03.855685 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:03.856313 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:03.857155 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:03.857160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:03.857415 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:03.857688 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:03.857694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.859407 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.859409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:03.860703 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:03.860706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:03.864492 (+ 3786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:03.864494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:03.864711 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:03.865360 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:03.865372 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:03.867226 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:03.874129 (+ 6903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:03.874142 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:03.874144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:03.874554 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:03.876476 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:03.876477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:03.876478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:03.876660 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:03.877225 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:03.877234 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:03.877422 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 21:11:03.877547 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 21:11:03.877817 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9240,"thread_start_us":103,"threads_started":1}
I20260812 21:11:03.879413 3633 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37783
I20260812 21:11:03.879542 3633 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:03.879779 3633 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
W20260812 21:11:03.880787 3633 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:37783 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 21:11:03.880875 3633 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:03.880956 3633 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:03.881286 3633 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:03.881374 3633 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
W20260812 21:11:03.881776 3633 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:11:03.966321 3637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:03.966528 3637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:03.966552 3637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:03.967707 3637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:03.967751 3637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:03.967777 3637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:03.969347 3637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:34287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=39825
--tserver_master_addrs=127.11.207.190:37783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:03.969583 3637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:03.969797 3637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:03.974977 3646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:03.974984 3648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:03.975839 3637 server_base.cc:1034] running on GCE node
W20260812 21:11:03.976693 3652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:03.976990 3637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:03.977257 3637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:03.978427 3637 hybrid_clock.cc:648] HybridClock initialized: now 1786569063978394 us; error 48 us; skew 500 ppm
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:03.981479 3637 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:03.981956 3637 webserver.cc:460] Webserver started at http://127.11.207.130:39825/ using document root <none> and password file <none>
I20260812 21:11:03.982220 3637 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:03.982291 3637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:11:03 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:03.999756 3637 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:04.001010 3667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:04.001228 3637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:04.001327 3637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "c9552a3bbee64587be7f9d3ec9dabe9b"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "bbc0dcb57b0acd3083c96ea0d4fa07dc"
server_key_iv: "1f70d4d11d7a8802fdf52c1707a50bef"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:04.001814 3637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:04.020154 3637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:04.022300 3637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:04.022454 3637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:04.022701 3637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:04.023155 3637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:04.023205 3637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:04.023237 3637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:04.023277 3637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:04.029855 3637 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:34287
I20260812 21:11:04.030257 3637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:04.030421 3901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:34287 every 8 connection(s)
I20260812 21:11:04.035390 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 3637
I20260812 21:11:04.035688 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.207.148:56913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:38693
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:37783
--webserver_port=41231
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:11:04 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:04.041064 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:04.031027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:45705 (local address 127.11.207.190:37783)
0812 21:11:04.031098 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:04.031100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:04.031285 (+ 185us) server_negotiation.cc:407] Connection header received
0812 21:11:04.031769 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:04.031770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:04.031790 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:04.031886 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:04.032445 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.033202 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.034202 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.034412 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.034469 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:04.037748 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:04.037750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:04.037750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:04.037760 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:04.039474 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:04.039947 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:04.039948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:04.039949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:04.040014 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:04.040286 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:04.040287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:04.040287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:04.040453 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:04.040495 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:04.040862 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 21:11:04.040966 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:11:04.041895 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:04.030984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37783 (local address 127.11.207.130:45705)
0812 21:11:04.031141 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:04.031182 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:04.031661 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:04.031942 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:04.031945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:04.032117 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:04.032322 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:04.032326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.033307 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.033307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:04.034109 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:04.034111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.034506 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.034507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:04.034626 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:04.035130 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:04.035138 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:04.037630 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:04.039566 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:04.039574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:04.039577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:04.039860 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:04.040078 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:04.040078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:04.040078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:04.040208 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:04.040538 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:04.040546 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:04.040715 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 21:11:04.041464 (+ 749us) client_negotiation.cc:240] Negotiation successful
0812 21:11:04.041647 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260812 21:11:04.042594 3904 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37783
I20260812 21:11:04.042695 3904 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.042898 3904 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
W20260812 21:11:04.043642 3904 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:37783 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 21:11:04.043710 3904 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.043772 3904 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:04.044134 3904 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.044202 3904 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
W20260812 21:11:04.044584 3904 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:11:04.123966 3919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:04.124182 3919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:04.124215 3919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:04.125393 3919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:04.125438 3919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:04.125474 3919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:04.127512 3919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.207.148:56913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:38693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=41231
--tserver_master_addrs=127.11.207.190:37783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:04.128011 3919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:04.128301 3919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:04.134887 3931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:04.135123 3934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:04.134887 3925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:04.135725 3919 server_base.cc:1034] running on GCE node
I20260812 21:11:04.136036 3919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:04.136543 3919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:04.137795 3919 hybrid_clock.cc:648] HybridClock initialized: now 1786569064137734 us; error 78 us; skew 500 ppm
Aug 12 21:11:04 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:04.141415 3919 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:04.142184 3919 webserver.cc:460] Webserver started at http://127.11.207.131:41231/ using document root <none> and password file <none>
I20260812 21:11:04.142515 3919 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:04.142627 3919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:11:04 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:04.158571 3919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:04.160013 3947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:04.160246 3919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:04.160386 3919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "956493b71299477980bb27f6f0b004f9"
format_stamp: "Formatted at 2026-08-12 21:11:03 on dist-test-slave-lkx2"
server_key: "759437f2fd45d72843218b3874efd0f7"
server_key_iv: "fe7665ccfaa8d1decdd9fe99c4595391"
server_key_version: "kuduclusterkey@0"
I20260812 21:11:04.160883 3919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:04.189144 3919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:04.191958 3919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:04.192205 3919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:04.192579 3919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:04.193041 3919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:04.193132 3919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:04.193207 3919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:04.193285 3919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:04.200536 3919 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:38693
I20260812 21:11:04.201100 3919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:11:04.204515 4160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:38693 every 8 connection(s)
I20260812 21:11:04.209286 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 3919
Aug 12 21:11:04 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:04.217006 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:04.202717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:55085 (local address 127.11.207.190:37783)
0812 21:11:04.202768 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:04.202770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:04.204878 (+ 2108us) server_negotiation.cc:407] Connection header received
0812 21:11:04.205541 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:04.205542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:04.205556 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:04.205654 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:04.206756 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.207565 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.209556 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.209860 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.209911 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:04.212609 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:04.212611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:04.212611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:04.212619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:04.214262 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:04.215867 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:04.215868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:04.215869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:04.215934 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:04.216199 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:04.216200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:04.216201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:04.216381 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:04.216435 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:04.216812 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 21:11:04.216904 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:11:04.217558 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:04.202430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37783 (local address 127.11.207.131:55085)
0812 21:11:04.204723 (+ 2293us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:04.204766 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:04.205409 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:04.206052 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:04.206057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:04.206319 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:04.206616 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:04.206623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.207681 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.207682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:04.208991 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:04.208995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.210057 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.210058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:04.210231 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:04.210822 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:04.210832 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:04.212496 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:04.214341 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:04.214348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:04.214350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:04.215782 (+ 1432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:04.215994 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:04.215994 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:04.215995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:04.216129 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:04.216467 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:04.216474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:04.216645 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 21:11:04.217142 (+ 497us) client_negotiation.cc:240] Negotiation successful
0812 21:11:04.217322 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2249,"thread_start_us":82,"threads_started":1}
I20260812 21:11:04.218101 4163 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37783
I20260812 21:11:04.218195 4163 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.218400 4163 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
W20260812 21:11:04.219045 4163 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:37783 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 21:11:04.219112 4163 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.219170 4163 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:04.219496 4163 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.219568 4163 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
W20260812 21:11:04.219862 4163 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:11:04.226442 4168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:04.209973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33366 (local address 127.11.207.190:37783)
0812 21:11:04.212647 (+ 2674us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:04.212649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:04.212656 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:11:04.212676 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:04.212677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:04.212689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:04.212738 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:04.213795 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.214487 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.220421 (+ 5934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:04.220754 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:04.221270 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:04.222372 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:04.222373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:04.222374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:04.222382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:04.224095 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:04.224704 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:04.224706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:04.224706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:04.224773 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:04.225779 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:04.225780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:04.225780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:04.225920 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:04.226089 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:04.226155 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 21:11:04.226250 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2653,"thread_start_us":82,"threads_started":1}
Aug 12 21:11:04 dist-test-slave-lkx2 krb5kdc[1397](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786569064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:04.881958 3633 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:04.882120 3633 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:04.882469 3287 ts_manager.cc:194] Registered new tserver with Master: e8e3e5120d4644ada84fc7ea8572a2f9 (127.11.207.129:34511)
I20260812 21:11:04.883173 3287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:57331
I20260812 21:11:05.044736 3904 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:05.044924 3904 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:05.045212 3287 ts_manager.cc:194] Registered new tserver with Master: c9552a3bbee64587be7f9d3ec9dabe9b (127.11.207.130:34287)
I20260812 21:11:05.045611 3287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:45705
Aug 12 21:11:05 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
I20260812 21:11:05.220041 4163 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:05.220211 4163 heartbeater.cc:511] Master 127.11.207.190:37783 requested a full tablet report, sending...
I20260812 21:11:05.220633 3287 ts_manager.cc:194] Registered new tserver with Master: 956493b71299477980bb27f6f0b004f9 (127.11.207.131:38693)
I20260812 21:11:05.221125 3287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:55085
I20260812 21:11:05.231644 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:11:05.248451 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.239918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.11.207.190:37783)
0812 21:11:05.240316 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.240318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.240326 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:11:05.240376 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.240377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.240389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.241159 (+ 770us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:05.241201 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.241979 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.243015 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.243337 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.244922 (+ 1585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.244982 (+ 60us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:05.244983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:05.244983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:05.244988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:05.246795 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:05.247213 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:05.247214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:05.247215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:05.247278 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:05.247495 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:05.247496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:05.247496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:05.247637 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:05.248179 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.248228 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.248329 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":74,"threads_started":1}
I20260812 21:11:05.250958 3287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:11:05.253738 3287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:11:05.273516 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.262717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:38693 (local address 127.0.0.1:46040)
0812 21:11:05.268602 (+ 5885us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.268656 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:05.268733 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:05.269031 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:05.269035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:05.269037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:05.269115 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.269116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.270694 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.270696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.272092 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.272096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.273191 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.273192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.273274 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.273287 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 21:11:05.273351 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:11:05.273418 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5851,"mutex_wait_us":43,"thread_start_us":78,"threads_started":1}
I20260812 21:11:05.274283 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.263205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.11.207.131:38693)
0812 21:11:05.268444 (+ 5239us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.268449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.268816 (+ 367us) server_negotiation.cc:407] Connection header received
0812 21:11:05.268853 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.268856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.268871 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.268931 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:05.269201 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.270317 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.272193 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.273016 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.273095 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.273154 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.274152 (+ 998us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.274198 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5200,"thread_start_us":53,"threads_started":1}
I20260812 21:11:05.275925 4038 tablet_service.cc:1467] Processing CreateTablet for tablet f82915658ffc4e05a302cfd23ddf138b (DEFAULT_TABLE table=test-table [id=9fee447566784f42a49b8cddd5ce3bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:05.276325 4038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82915658ffc4e05a302cfd23ddf138b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:05.280411 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.262728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:34287 (local address 127.0.0.1:37116)
0812 21:11:05.269227 (+ 6499us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.269236 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:05.269276 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:05.269526 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:05.269527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:05.269529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:05.269562 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.269564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.270683 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.270684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.273725 (+ 3041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.273728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.276197 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.276198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.276259 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.280188 (+ 3929us) client_negotiation.cc:769] Sending connection context
0812 21:11:05.280250 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 21:11:05.280300 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6482,"mutex_wait_us":5772,"thread_start_us":99,"threads_started":1}
I20260812 21:11:05.280453 4219 tablet_bootstrap.cc:492] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Bootstrap starting.
I20260812 21:11:05.281121 4219 tablet_bootstrap.cc:654] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:05.281464 4219 log.cc:824] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:05.281759 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.263335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.11.207.130:34287)
0812 21:11:05.268522 (+ 5187us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.268526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.269326 (+ 800us) server_negotiation.cc:407] Connection header received
0812 21:11:05.269364 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.269367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.269382 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.269442 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:05.269620 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.270591 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.275364 (+ 4773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.276098 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.276282 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.276387 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.281608 (+ 5221us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.281669 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5156,"thread_start_us":52,"threads_started":1}
I20260812 21:11:05.282289 4219 tablet_bootstrap.cc:492] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: No bootstrap required, opened a new log
I20260812 21:11:05.282370 4219 ts_tablet_manager.cc:1397] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:11:05.283072 3743 tablet_service.cc:1467] Processing CreateTablet for tablet f82915658ffc4e05a302cfd23ddf138b (DEFAULT_TABLE table=test-table [id=9fee447566784f42a49b8cddd5ce3bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:05.283463 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82915658ffc4e05a302cfd23ddf138b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:05.283913 4216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.263072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.11.207.129:34511)
0812 21:11:05.274007 (+ 10935us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.274012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.274023 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:11:05.274059 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.274061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.274077 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.274285 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:05.274395 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.275118 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.282578 (+ 7460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.283403 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.283458 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.283511 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.283772 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.283831 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10891,"thread_start_us":74,"threads_started":1}
I20260812 21:11:05.284147 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.262934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:34511 (local address 127.0.0.1:39054)
0812 21:11:05.273837 (+ 10903us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.273847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:05.273887 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:05.274172 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:05.274173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:05.274175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:05.274213 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.274215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.275198 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.275199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.282328 (+ 7129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.282467 (+ 139us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.283575 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.283576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.283641 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.283645 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:05.284018 (+ 373us) client_negotiation.cc:240] Negotiation successful
0812 21:11:05.284068 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10886,"thread_start_us":62,"threads_started":1}
I20260812 21:11:05.284860 3493 tablet_service.cc:1467] Processing CreateTablet for tablet f82915658ffc4e05a302cfd23ddf138b (DEFAULT_TABLE table=test-table [id=9fee447566784f42a49b8cddd5ce3bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:05.284957 4219 raft_consensus.cc:348] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.285115 4219 raft_consensus.cc:374] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:05.285149 4219 raft_consensus.cc:729] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956493b71299477980bb27f6f0b004f9, State: Initialized, Role: FOLLOWER
I20260812 21:11:05.285207 3493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82915658ffc4e05a302cfd23ddf138b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:05.285266 4219 consensus_queue.cc:260] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.285528 4219 ts_tablet_manager.cc:1428] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 21:11:05.287743 4224 tablet_bootstrap.cc:492] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Bootstrap starting.
I20260812 21:11:05.288471 4224 tablet_bootstrap.cc:654] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:05.288475 4163 heartbeater.cc:503] Master 127.11.207.190:37783 was elected leader, sending a full tablet report...
I20260812 21:11:05.288836 4224 log.cc:824] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:05.289630 4224 tablet_bootstrap.cc:492] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: No bootstrap required, opened a new log
I20260812 21:11:05.289700 4224 ts_tablet_manager.cc:1397] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:11:05.289994 4226 tablet_bootstrap.cc:492] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: Bootstrap starting.
I20260812 21:11:05.290623 4226 tablet_bootstrap.cc:654] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:05.290939 4226 log.cc:824] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:05.291693 4226 tablet_bootstrap.cc:492] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: No bootstrap required, opened a new log
I20260812 21:11:05.291756 4226 ts_tablet_manager.cc:1397] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:11:05.292297 4224 raft_consensus.cc:348] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.292668 4224 raft_consensus.cc:374] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:05.292755 4224 raft_consensus.cc:729] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9552a3bbee64587be7f9d3ec9dabe9b, State: Initialized, Role: FOLLOWER
I20260812 21:11:05.292950 4224 consensus_queue.cc:260] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.293452 4224 ts_tablet_manager.cc:1428] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260812 21:11:05.293618 3904 heartbeater.cc:503] Master 127.11.207.190:37783 was elected leader, sending a full tablet report...
I20260812 21:11:05.294473 4226 raft_consensus.cc:348] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.294641 4226 raft_consensus.cc:374] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:05.294674 4226 raft_consensus.cc:729] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e3e5120d4644ada84fc7ea8572a2f9, State: Initialized, Role: FOLLOWER
I20260812 21:11:05.294827 4226 consensus_queue.cc:260] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.295109 4226 ts_tablet_manager.cc:1428] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 21:11:05.295253 3633 heartbeater.cc:503] Master 127.11.207.190:37783 was elected leader, sending a full tablet report...
W20260812 21:11:05.356643 3634 tablet.cc:2369] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 21:11:05 dist-test-slave-lkx2 krb5kdc[1397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569064, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for HTTP/127.11.207.148@KRBTEST.COM
W20260812 21:11:05.453707 4164 tablet.cc:2369] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:05.456039 4223 raft_consensus.cc:482] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:05.456174 4223 raft_consensus.cc:504] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.456634 4223 leader_election.cc:288] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9552a3bbee64587be7f9d3ec9dabe9b (127.11.207.130:34287), e8e3e5120d4644ada84fc7ea8572a2f9 (127.11.207.129:34511)
I20260812 21:11:05.463208 4216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.457096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:39485 (local address 127.11.207.129:34511)
0812 21:11:05.457143 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.457145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.459363 (+ 2218us) server_negotiation.cc:407] Connection header received
0812 21:11:05.459455 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.459457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.459467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.459496 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:05.459723 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.460539 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.462093 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.462702 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.462753 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.462763 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.463070 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.463128 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:11:05.463692 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.456788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:34511 (local address 127.11.207.131:39485)
0812 21:11:05.459310 (+ 2522us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.459318 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:05.459385 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:05.459561 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:05.459562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:05.459564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:05.459645 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.459647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.460693 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.460693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.461976 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.461980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.462850 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.462851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.462931 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.462935 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:05.463540 (+ 605us) client_negotiation.cc:240] Negotiation successful
0812 21:11:05.463591 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2508,"thread_start_us":134,"threads_started":1}
I20260812 21:11:05.464183 3546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82915658ffc4e05a302cfd23ddf138b" candidate_uuid: "956493b71299477980bb27f6f0b004f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" is_pre_election: true
I20260812 21:11:05.464393 3546 raft_consensus.cc:2454] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956493b71299477980bb27f6f0b004f9 in term 0.
I20260812 21:11:05.464743 3948 leader_election.cc:302] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956493b71299477980bb27f6f0b004f9, e8e3e5120d4644ada84fc7ea8572a2f9; no voters:
I20260812 21:11:05.464887 4223 raft_consensus.cc:2781] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:11:05.464979 4223 raft_consensus.cc:482] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:11:05.465008 4223 raft_consensus.cc:3037] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:05.466006 4223 raft_consensus.cc:504] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.466158 4223 leader_election.cc:288] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [CANDIDATE]: Term 1 election: Requested vote from peers c9552a3bbee64587be7f9d3ec9dabe9b (127.11.207.130:34287), e8e3e5120d4644ada84fc7ea8572a2f9 (127.11.207.129:34511)
I20260812 21:11:05.466428 3546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82915658ffc4e05a302cfd23ddf138b" candidate_uuid: "956493b71299477980bb27f6f0b004f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9"
I20260812 21:11:05.466511 3546 raft_consensus.cc:3037] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:05.467440 3546 raft_consensus.cc:2454] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956493b71299477980bb27f6f0b004f9 in term 1.
I20260812 21:11:05.467743 3948 leader_election.cc:302] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956493b71299477980bb27f6f0b004f9, e8e3e5120d4644ada84fc7ea8572a2f9; no voters:
I20260812 21:11:05.467872 4223 raft_consensus.cc:2781] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:05.468060 4223 raft_consensus.cc:686] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 1 LEADER]: Becoming Leader. State: Replica: 956493b71299477980bb27f6f0b004f9, State: Running, Role: LEADER
I20260812 21:11:05.468164 4223 consensus_queue.cc:237] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } }
I20260812 21:11:05.469146 3284 catalog_manager.cc:5495] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 956493b71299477980bb27f6f0b004f9 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "956493b71299477980bb27f6f0b004f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "956493b71299477980bb27f6f0b004f9" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 38693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 } health_report { overall_health: UNKNOWN } } }
I20260812 21:11:05.470283 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.457049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:37417 (local address 127.11.207.130:34287)
0812 21:11:05.457097 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.457098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.460408 (+ 3310us) server_negotiation.cc:407] Connection header received
0812 21:11:05.460446 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.460447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.460458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.460520 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:05.466173 (+ 5653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.466825 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.469336 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.469874 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.469927 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.469937 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.470159 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.470201 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:11:05.470937 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.456774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:34287 (local address 127.11.207.131:37417)
0812 21:11:05.459134 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.459147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:05.459232 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:05.465991 (+ 6759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:05.465993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:05.465995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:05.466087 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.466089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.468157 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.468158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.469191 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:05.469194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.470004 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.470005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:05.470080 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.470085 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:11:05.470794 (+ 709us) client_negotiation.cc:240] Negotiation successful
0812 21:11:05.470845 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2337,"thread_start_us":141,"threads_started":1}
I20260812 21:11:05.471429 3804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82915658ffc4e05a302cfd23ddf138b" candidate_uuid: "956493b71299477980bb27f6f0b004f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" is_pre_election: true
I20260812 21:11:05.471603 3804 raft_consensus.cc:2454] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956493b71299477980bb27f6f0b004f9 in term 0.
I20260812 21:11:05.471781 3801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82915658ffc4e05a302cfd23ddf138b" candidate_uuid: "956493b71299477980bb27f6f0b004f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b"
I20260812 21:11:05.471863 3801 raft_consensus.cc:3037] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:05.472813 3801 raft_consensus.cc:2454] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956493b71299477980bb27f6f0b004f9 in term 1.
I20260812 21:11:05.499614 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.496258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.11.207.131:38693)
0812 21:11:05.496304 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:05.496306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:05.496315 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:11:05.496404 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:05.496405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:05.496428 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:05.496506 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:05.496673 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.497410 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.498514 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:05.498814 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:05.498874 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:05.499063 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:05.499200 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:11:05.499289 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:05.499505 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 21:11:05.499529 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:11:05.502190 3546 raft_consensus.cc:1261] T f82915658ffc4e05a302cfd23ddf138b P e8e3e5120d4644ada84fc7ea8572a2f9 [term 1 FOLLOWER]: Refusing update from remote peer 956493b71299477980bb27f6f0b004f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:05.502734 4235 consensus_queue.cc:1035] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8e3e5120d4644ada84fc7ea8572a2f9" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:05.505784 3801 raft_consensus.cc:1261] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 1 FOLLOWER]: Refusing update from remote peer 956493b71299477980bb27f6f0b004f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:05.506250 4235 consensus_queue.cc:1035] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9552a3bbee64587be7f9d3ec9dabe9b" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 34287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:05.511165 4241 mvcc.cc:205] Tried to move back new op lower bound from 7317786892293181440 to 7317786892158308352. Current Snapshot: MvccSnapshot[applied={T|T < 7317786892293181440}]
I20260812 21:11:05.511454 4249 mvcc.cc:205] Tried to move back new op lower bound from 7317786892293181440 to 7317786892158308352. Current Snapshot: MvccSnapshot[applied={T|T < 7317786892293181440}]
I20260812 21:11:05.514519 3284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:05.514618 3284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:05.515583 3284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7e495d7882741e68d3d26097ade29a2: Sending DeleteTablet for 3 replicas of tablet f82915658ffc4e05a302cfd23ddf138b
I20260812 21:11:05.515990 3493 tablet_service.cc:1514] Processing DeleteTablet for tablet f82915658ffc4e05a302cfd23ddf138b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:05 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:39054
I20260812 21:11:05.516149 3743 tablet_service.cc:1514] Processing DeleteTablet for tablet f82915658ffc4e05a302cfd23ddf138b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:05 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:37116
I20260812 21:11:05.516492 4038 tablet_service.cc:1514] Processing DeleteTablet for tablet f82915658ffc4e05a302cfd23ddf138b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:05 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:46040
I20260812 21:11:05.516866 4257 tablet_replica.cc:330] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: stopping tablet replica
I20260812 21:11:05.517040 4257 raft_consensus.cc:2229] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:05.517228 4257 raft_consensus.cc:2258] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:05.517264 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 3345
I20260812 21:11:05.533766 4257 ts_tablet_manager.cc:1905] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:05.535961 4257 ts_tablet_manager.cc:1918] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:05.536207 4259 tablet_replica.cc:330] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: stopping tablet replica
I20260812 21:11:05.536401 4257 log.cc:1196] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/f82915658ffc4e05a302cfd23ddf138b
I20260812 21:11:05.536607 4259 raft_consensus.cc:2229] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:11:05.537002 4257 ts_tablet_manager.cc:1939] T f82915658ffc4e05a302cfd23ddf138b P c9552a3bbee64587be7f9d3ec9dabe9b: Deleting consensus metadata
I20260812 21:11:05.537218 4259 raft_consensus.cc:2258] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:05.537956 3248 catalog_manager.cc:4841] TS c9552a3bbee64587be7f9d3ec9dabe9b (127.11.207.130:34287): tablet f82915658ffc4e05a302cfd23ddf138b (table test-table [id=9fee447566784f42a49b8cddd5ce3bb5]) successfully deleted
I20260812 21:11:05.538394 4259 ts_tablet_manager.cc:1905] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 21:11:05.538775 3948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:34511 (error 108)
W20260812 21:11:05.539294 3243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:34511 (error 108)
I20260812 21:11:05.539973 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 3637
I20260812 21:11:05.540107 4259 ts_tablet_manager.cc:1918] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:05.540165 4259 log.cc:1196] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/f82915658ffc4e05a302cfd23ddf138b
I20260812 21:11:05.540398 4259 ts_tablet_manager.cc:1939] T f82915658ffc4e05a302cfd23ddf138b P 956493b71299477980bb27f6f0b004f9: Deleting consensus metadata
I20260812 21:11:05.560683 3243 catalog_manager.cc:4841] TS 956493b71299477980bb27f6f0b004f9 (127.11.207.131:38693): tablet f82915658ffc4e05a302cfd23ddf138b (table test-table [id=9fee447566784f42a49b8cddd5ce3bb5]) successfully deleted
I20260812 21:11:05.560962 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.560608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:34511 (local address 127.0.0.1:39066)
0812 21:11:05.560869 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.560929 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:34511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
W20260812 21:11:05.561324 3243 catalog_manager.cc:4584] TS e8e3e5120d4644ada84fc7ea8572a2f9 (127.11.207.129:34511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f82915658ffc4e05a302cfd23ddf138b: Network error: Client connection negotiation failed: client connection to 127.11.207.129:34511: connect: Connection refused (error 111)
I20260812 21:11:05.561447 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:05.559290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:34511 (local address 127.11.207.131:39697)
0812 21:11:05.561352 (+ 2062us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:05.561408 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:34511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2027}
I20260812 21:11:05.566913 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 3919
I20260812 21:11:05.579782 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 3213
I20260812 21:11:05.595067 12094 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 21:11:05.946600 12094 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 21:11:05.946694 12094 mini_ranger.cc:66] Stopping Ranger...
I20260812 21:11:06.301198 12094 mini_ranger.cc:68] Stopped Ranger
2026-08-12 21:11:06.301 UTC [1454] LOG: received smart shutdown request
2026-08-12 21:11:06.305 UTC [1454] LOG: background worker "logical replication launcher" (PID 1467) exited with exit code 1
2026-08-12 21:11:06.306 UTC [1459] LOG: shutting down
2026-08-12 21:11:06.458 UTC [1454] LOG: database system is shut down
2026-08-12T21:11:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49839 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:11:06 dist-test-slave-lkx2 krb5kdc[4275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:11:06 dist-test-slave-lkx2 krb5kdc[4275](info): set up 2 sockets
Aug 12 21:11:06 dist-test-slave-lkx2 krb5kdc[4275](info): commencing operation
krb5kdc: starting...
W20260812 21:11:07.756156 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:11:07 dist-test-slave-lkx2 krb5kdc[4275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:11:07.879000 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:39911
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:39911,127.11.207.189:41113,127.11.207.188:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:07.961728 4308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:07.961959 4308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:07.961992 4308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:07.963133 4308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:07.963176 4308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:07.963194 4308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:07.963212 4308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:07.963227 4308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:07.964772 4308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:39911,127.11.207.189:41113,127.11.207.188:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:07.965022 4308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:07.965230 4308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:07.969950 4325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:07.969991 4308 server_base.cc:1034] running on GCE node
W20260812 21:11:07.969965 4319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:07.969961 4317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:07.970538 4308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:07.970830 4308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:07.971984 4308 hybrid_clock.cc:648] HybridClock initialized: now 1786569067971966 us; error 29 us; skew 500 ppm
Aug 12 21:11:07 dist-test-slave-lkx2 krb5kdc[4275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:07.974542 4308 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:07.974995 4308 webserver.cc:460] Webserver started at http://127.11.207.190:35199/ using document root <none> and password file <none>
I20260812 21:11:07.975203 4308 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:07.975251 4308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:07.975386 4308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:07.976502 4308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "970e33efcdd648048eb35e158bb32134"
format_stamp: "Formatted at 2026-08-12 21:11:07 on dist-test-slave-lkx2"
server_key: "07bdf9a70c47facf2c3f93bc544d4787"
server_key_iv: "33b55bd4840c0844181357f927fb14d5"
server_key_version: "encryptionkey@0"
I20260812 21:11:07.976846 4308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "970e33efcdd648048eb35e158bb32134"
format_stamp: "Formatted at 2026-08-12 21:11:07 on dist-test-slave-lkx2"
server_key: "07bdf9a70c47facf2c3f93bc544d4787"
server_key_iv: "33b55bd4840c0844181357f927fb14d5"
server_key_version: "encryptionkey@0"
I20260812 21:11:07.978137 4308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:07.978790 4336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:07.978947 4308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:07.979043 4308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "970e33efcdd648048eb35e158bb32134"
format_stamp: "Formatted at 2026-08-12 21:11:07 on dist-test-slave-lkx2"
server_key: "07bdf9a70c47facf2c3f93bc544d4787"
server_key_iv: "33b55bd4840c0844181357f927fb14d5"
server_key_version: "encryptionkey@0"
I20260812 21:11:07.979108 4308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:08.021031 4308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:08.023372 4308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:08.023523 4308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:08.027632 4308 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:39911
I20260812 21:11:08.027678 4430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:39911 every 8 connection(s)
I20260812 21:11:08.028035 4308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:08.028970 4432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:08.030570 4432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.034789 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 4308
I20260812 21:11:08.034904 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:11:08.035101 12094 external_mini_cluster.cc:1428] Setting key 2d97d38d266dd0e50615b9967e676dad
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:08.040576 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.031364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.11.207.190:39911)
0812 21:11:08.031512 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.031522 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.031631 (+ 109us) server_negotiation.cc:407] Connection header received
0812 21:11:08.032182 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.032184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.032201 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.032283 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.032854 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.033731 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.034592 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.035073 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.035195 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.037400 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.037408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.037409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.037417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.038960 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.039311 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.039314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.039315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.039395 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.039686 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.039687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.039687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.039919 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.039983 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.040222 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.040369 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260812 21:11:08.040589 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.031285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:39911 (local address 127.0.0.1:49974)
0812 21:11:08.031488 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.031536 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.032026 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.032302 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.032305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.032598 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.032763 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.032767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.033824 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.033825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.034515 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.034517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.035172 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.035174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.035253 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.035743 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.035752 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.037319 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.039041 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.039048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.039050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.039241 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.039484 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.039485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.039486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.039613 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.039987 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.039995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.040079 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.040191 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.040455 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":83,"thread_start_us":83,"threads_started":1}
I20260812 21:11:08.041395 4432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.041744 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.041607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47952)
0812 21:11:08.041663 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.041718 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 21:11:08.041891 4338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
I20260812 21:11:08.042320 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.042219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47962)
0812 21:11:08.042265 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.042297 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 21:11:08.042455 4432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:41113: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.207.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab.
Entry for principal kudu/127.11.207.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab.
I20260812 21:11:08.062655 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:41113
--webserver_interface=127.11.207.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab
--principal=kudu/127.11.207.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:39911,127.11.207.189:41113,127.11.207.188:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:08.091679 4432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } attempt: 1
I20260812 21:11:08.092206 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.092075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47964)
0812 21:11:08.092132 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.092175 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 21:11:08.092562 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.092486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47966)
0812 21:11:08.092520 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.092543 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:11:08.092651 4432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:41113: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
W20260812 21:11:08.147501 4447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:08.147745 4447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:08.147770 4447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:08.149005 4447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:08.149050 4447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:08.149063 4447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:08.149075 4447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:08.149086 4447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:08.150583 4447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:39911,127.11.207.189:41113,127.11.207.188:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab
--principal=kudu/127.11.207.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:41113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.207.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:08.150810 4447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:08.150995 4447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:08.155949 4461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:08.155900 4463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:08.155915 4460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:08.155968 4447 server_base.cc:1034] running on GCE node
I20260812 21:11:08.156383 4447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:08.156704 4447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:08.157847 4447 hybrid_clock.cc:648] HybridClock initialized: now 1786569068157827 us; error 38 us; skew 500 ppm
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:08.160378 4447 init.cc:377] Logged in from keytab as kudu/127.11.207.189@KRBTEST.COM (short username kudu)
I20260812 21:11:08.160818 4447 webserver.cc:460] Webserver started at http://127.11.207.189:41959/ using document root <none> and password file <none>
I20260812 21:11:08.160813 4432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } attempt: 2
I20260812 21:11:08.161052 4447 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:08.161106 4447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:08.161209 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.161070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47978)
0812 21:11:08.161133 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.161175 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:11:08.161229 4447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:08.161502 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.161408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47986)
0812 21:11:08.161452 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.161481 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 21:11:08.161587 4432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:41113: Network error: Client connection negotiation failed: client connection to 127.11.207.189:41113: connect: Connection refused (error 111)
I20260812 21:11:08.162395 4447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data/instance:
uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16"
format_stamp: "Formatted at 2026-08-12 21:11:08 on dist-test-slave-lkx2"
server_key: "7419bc8654eb7678c149e50327660438"
server_key_iv: "c4b39c0e81bd56dbcd640e7f9c64cc88"
server_key_version: "encryptionkey@0"
I20260812 21:11:08.162688 4447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal/instance:
uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16"
format_stamp: "Formatted at 2026-08-12 21:11:08 on dist-test-slave-lkx2"
server_key: "7419bc8654eb7678c149e50327660438"
server_key_iv: "c4b39c0e81bd56dbcd640e7f9c64cc88"
server_key_version: "encryptionkey@0"
I20260812 21:11:08.163929 4447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:11:08.164690 4474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:08.164872 4447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:08.164968 4447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal
uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16"
format_stamp: "Formatted at 2026-08-12 21:11:08 on dist-test-slave-lkx2"
server_key: "7419bc8654eb7678c149e50327660438"
server_key_iv: "c4b39c0e81bd56dbcd640e7f9c64cc88"
server_key_version: "encryptionkey@0"
I20260812 21:11:08.165030 4447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:08.182969 4447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:08.184926 4447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:08.185071 4447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:08.189143 4447 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.189:41113
I20260812 21:11:08.189189 4572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.189:41113 every 8 connection(s)
I20260812 21:11:08.189539 4447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
I20260812 21:11:08.190402 4573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:08.192052 4573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:08.198798 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 4447
I20260812 21:11:08.198908 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-1/wal/instance
I20260812 21:11:08.199082 12094 external_mini_cluster.cc:1428] Setting key 5e3396ac7ec15c52eb63cf290d4c2e12
I20260812 21:11:08.200757 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.192801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49988 (local address 127.11.207.190:39911)
0812 21:11:08.192895 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.192898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.193083 (+ 185us) server_negotiation.cc:407] Connection header received
0812 21:11:08.193570 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.193571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.193583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.193624 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.194218 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.194708 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.195676 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.195885 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.195931 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.197996 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.197997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.197998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.198005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.199397 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.199785 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.199787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.199787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.199850 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.200080 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.200081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.200082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.200202 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.200258 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.200569 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.200665 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 21:11:08.200932 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.192768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:39911 (local address 127.0.0.1:49988)
0812 21:11:08.192960 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.192999 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.193477 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.193690 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.193693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.193955 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.194154 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.194158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.194798 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.194799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.195612 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.195615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.195939 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.195939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.196036 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.196546 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.196555 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.197899 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.199475 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.199483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.199485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.199714 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.199911 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.199912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.199913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.200025 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.200258 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.200266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.200450 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.200541 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.200703 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260812 21:11:08.201453 4573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.188@KRBTEST.COM; defaulting to no policy
I20260812 21:11:08.208945 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.201704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:47990)
0812 21:11:08.201752 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.201763 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.201806 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.202056 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.202057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.202189 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.202236 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.202237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.203349 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.203350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.203847 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.203848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.204512 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.204513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.204569 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.204962 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.204967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.206094 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.207925 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.207926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.207926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.208076 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.208280 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.208281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.208281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.208335 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.208682 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.208682 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.208747 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.208801 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.208878 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:11:08.209049 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.201768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47990 (local address 127.11.207.189:41113)
0812 21:11:08.201927 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.201931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.201949 (+ 18us) server_negotiation.cc:407] Connection header received
0812 21:11:08.201967 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.201969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.201981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.202044 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.202298 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.203254 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.203928 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.204443 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.204516 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.206170 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.206180 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.206180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.206188 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.207828 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.208149 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.208153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.208153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.208219 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.208434 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.208435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.208435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.208621 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.208706 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.208834 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.208940 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260812 21:11:08.209631 4573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.209941 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.209837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56482)
0812 21:11:08.209878 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.209923 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:11:08.210069 4475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
I20260812 21:11:08.210466 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.210396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56492)
0812 21:11:08.210428 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.210451 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 21:11:08.210554 4573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:41405: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab.
Entry for principal kudu/127.11.207.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab.
I20260812 21:11:08.225224 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:41405
--webserver_interface=127.11.207.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab
--principal=kudu/127.11.207.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:39911,127.11.207.189:41113,127.11.207.188:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:08.252730 4432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } attempt: 3
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:11:08.259604 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.253233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48002 (local address 127.11.207.189:41113)
0812 21:11:08.253298 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.253300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.253310 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:11:08.253341 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.253341 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.253353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.253410 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.253779 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.254297 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.254980 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.255192 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.255257 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.257256 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.257258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.257258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.257266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.258548 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.258891 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.258892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.258893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.258947 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.259108 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.259109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.259109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.259223 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.259279 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.259440 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.259516 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:11:08.259604 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.253150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:48002)
0812 21:11:08.253223 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.253233 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.253285 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.253428 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.253429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.253591 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.253659 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.253661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.254373 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.254375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.254905 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.254907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.255262 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.255264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.255336 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.255741 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.255746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.257160 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.258627 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.258629 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.258629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.258825 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.259001 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.259002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.259002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.259059 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.259278 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.259279 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.259329 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.259390 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.259495 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 21:11:08.259903 4573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } attempt: 1
I20260812 21:11:08.260265 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.260157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56508)
0812 21:11:08.260205 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.260239 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:11:08.260485 4432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.260639 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.260547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56512)
0812 21:11:08.260581 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.260612 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 21:11:08.260743 4573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:41405: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
I20260812 21:11:08.260807 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.260713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56524)
0812 21:11:08.260754 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.260779 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 21:11:08.261106 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.261033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56528)
0812 21:11:08.261071 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.261092 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 21:11:08.261189 4432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:41405: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
I20260812 21:11:08.292369 4432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } attempt: 1
I20260812 21:11:08.292866 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.292689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56544)
0812 21:11:08.292758 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.292812 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 21:11:08.293318 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.293217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56550)
0812 21:11:08.293261 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.293292 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 21:11:08.293450 4432 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:41405: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
W20260812 21:11:08.309574 4585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:08.309783 4585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:08.309804 4585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:08.310968 4585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:08.311013 4585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:08.311026 4585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:08.311038 4585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:08.311062 4585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:08.312619 4585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:39911,127.11.207.189:41113,127.11.207.188:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab
--principal=kudu/127.11.207.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.207.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:08.312862 4585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:08.313091 4585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:08.317772 4596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:08.317940 4585 server_base.cc:1034] running on GCE node
W20260812 21:11:08.317800 4592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:08.317797 4593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:08.318302 4585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:08.318589 4585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:08.319736 4585 hybrid_clock.cc:648] HybridClock initialized: now 1786569068319719 us; error 57 us; skew 500 ppm
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:08.322252 4585 init.cc:377] Logged in from keytab as kudu/127.11.207.188@KRBTEST.COM (short username kudu)
I20260812 21:11:08.322718 4585 webserver.cc:460] Webserver started at http://127.11.207.188:32807/ using document root <none> and password file <none>
I20260812 21:11:08.322953 4585 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:08.323002 4585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:08.323132 4585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:08.324208 4585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data/instance:
uuid: "378387485b7f468aba661087ff530f05"
format_stamp: "Formatted at 2026-08-12 21:11:08 on dist-test-slave-lkx2"
server_key: "f1a2647574734fb19750b1d8827e4140"
server_key_iv: "b25702d3fb21c83378874aa1a0307e35"
server_key_version: "encryptionkey@0"
I20260812 21:11:08.324581 4585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal/instance:
uuid: "378387485b7f468aba661087ff530f05"
format_stamp: "Formatted at 2026-08-12 21:11:08 on dist-test-slave-lkx2"
server_key: "f1a2647574734fb19750b1d8827e4140"
server_key_iv: "b25702d3fb21c83378874aa1a0307e35"
server_key_version: "encryptionkey@0"
I20260812 21:11:08.325892 4585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:08.326601 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:08.326768 4585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:08.326869 4585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal
uuid: "378387485b7f468aba661087ff530f05"
format_stamp: "Formatted at 2026-08-12 21:11:08 on dist-test-slave-lkx2"
server_key: "f1a2647574734fb19750b1d8827e4140"
server_key_iv: "b25702d3fb21c83378874aa1a0307e35"
server_key_version: "encryptionkey@0"
I20260812 21:11:08.326931 4585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:08.328904 4573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } attempt: 2
I20260812 21:11:08.329312 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.329201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56552)
0812 21:11:08.329255 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.329285 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:11:08.329636 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.329547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56564)
0812 21:11:08.329588 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.329614 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:11:08.329846 4573 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:41405: Network error: Client connection negotiation failed: client connection to 127.11.207.188:41405: connect: Connection refused (error 111)
I20260812 21:11:08.338548 4585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:08.340500 4585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:08.340626 4585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:08.344941 4585 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.188:41405
I20260812 21:11:08.344988 4709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.188:41405 every 8 connection(s)
I20260812 21:11:08.345348 4585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
I20260812 21:11:08.346256 4710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:08.347832 4710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.350867 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 4585
I20260812 21:11:08.350979 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568904794702-12094-0/minicluster-data/master-2/wal/instance
I20260812 21:11:08.351147 12094 external_mini_cluster.cc:1428] Setting key db884e5f5e59659bbd7a9bf2a8546b6a
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:11:08.356997 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.348616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.11.207.190:39911)
0812 21:11:08.348740 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.348742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.348873 (+ 131us) server_negotiation.cc:407] Connection header received
0812 21:11:08.349312 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.349313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.349323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.349378 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.349981 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.350459 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.351553 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.351858 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.351933 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.354248 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.354249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.354250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.354257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.355716 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.356085 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.356087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.356087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.356145 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.356378 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.356379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.356379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.356512 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.356576 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.356833 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.356919 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 21:11:08.357146 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.348553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:39911 (local address 127.0.0.1:50004)
0812 21:11:08.348734 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.348773 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.349233 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.349431 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.349434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.349685 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.349902 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.349906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.350524 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.350524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.351431 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.351434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.351949 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.351950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.352111 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.352628 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.352638 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.354158 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.355789 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.355796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.355797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.356008 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.356198 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.356199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.356199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.356301 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.356572 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.356579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.356697 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.356791 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.356921 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260812 21:11:08.357808 4710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.359855 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.351517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.11.207.188:41405)
0812 21:11:08.351708 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.351714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.351751 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:11:08.351794 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.351796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.351817 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.351907 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.352619 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.353290 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.354163 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.354518 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.354584 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.356912 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.356919 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.356920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.356926 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.358689 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.358978 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.358981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.358981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.359039 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.359216 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.359216 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.359217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.359380 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.359539 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.359580 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.359751 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:11:08.364487 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.358047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:41113 (local address 127.0.0.1:48006)
0812 21:11:08.358087 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.358098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.358136 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.358268 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.358269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.358420 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.358456 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.358457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.359027 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.359028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.359719 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.359722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.360082 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.360083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.360318 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.360623 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.360627 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.361821 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.363479 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.363480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.363480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.363659 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.363855 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.363856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.363856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.363917 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.364161 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.364162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.364208 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.364273 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.364392 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:11:08.364502 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.358122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.11.207.189:41113)
0812 21:11:08.358163 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.358165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.358172 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:11:08.358196 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.358197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.358205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.358252 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.358515 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.358976 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.359791 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.360014 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.360071 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.361893 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.361894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.361895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.361902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.363399 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.363733 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.363735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.363735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.363796 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.363972 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.363973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.363973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.364088 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.364149 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.364309 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.364413 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:08.365029 4710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:11:08.367606 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.360790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.11.207.190:39911)
0812 21:11:08.360844 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.360846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.360853 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:11:08.360871 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.360872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.360880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.360921 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.361459 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.361970 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.362716 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.362997 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.363089 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.365047 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.365048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.365049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.365056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.366461 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.366835 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.366836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.366837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.366894 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.367097 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.367098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.367099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.367243 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.367432 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.367467 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.367531 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:11:08.368592 4432 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } attempt: 2
I20260812 21:11:08.372160 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.365290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56584)
0812 21:11:08.365319 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.365328 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.365363 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.365469 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.365470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.365586 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.365610 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.365612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.366170 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.366171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.366629 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.366630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.366966 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.366967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.367010 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.367362 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.367366 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.368648 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.371172 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.371173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.371173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.371351 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.371533 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.371534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.371534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.371613 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.371863 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.371864 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.371901 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.371970 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.372059 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 21:11:08.372426 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.365324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.11.207.188:41405)
0812 21:11:08.365357 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.365359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.365366 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:11:08.365412 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.365412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.365421 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.365458 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.365667 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.366107 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.366712 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.366915 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.366964 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.368742 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.368743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.368743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.368750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.371106 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.371418 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.371419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.371419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.371461 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.371681 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.371682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.371683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.371810 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.371868 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.371993 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.372106 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":19}
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569067, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:11:08.374444 4710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: Bootstrap starting.
I20260812 21:11:08.375156 4710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:08.375512 4710 log.cc:824] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:08.376469 4710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: No bootstrap required, opened a new log
I20260812 21:11:08.377528 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.368550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.11.207.189:41113)
0812 21:11:08.368592 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.368594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.368935 (+ 341us) server_negotiation.cc:407] Connection header received
0812 21:11:08.369038 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.369040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.369050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.369081 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.369691 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.370387 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.371280 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.371552 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.371609 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.374655 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.374656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.374656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.374663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.376193 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.376596 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.376597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.376597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.376658 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.376882 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.376883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.376884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.377002 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.377149 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.377365 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.377437 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:11:08.378412 4710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.378574 4710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:08.378623 4710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378387485b7f468aba661087ff530f05, State: Initialized, Role: FOLLOWER
I20260812 21:11:08.378769 4710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.378870 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.368835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56600)
0812 21:11:08.368884 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.368895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.368932 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.369332 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.369334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.369485 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.369528 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.369530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.370722 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.370723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.371420 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.371423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.372259 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.372260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.372326 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.372727 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.372732 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.376022 (+ 3290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.377862 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.377863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.377863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.378061 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.378236 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.378236 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.378237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.378294 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.378596 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.378597 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.378645 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.378701 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.378797 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:11:08.379127 4710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:08.379185 4722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.379287 4722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:11:08.379444 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:11:08.380600 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.369027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.11.207.188:41405)
0812 21:11:08.369201 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.369202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.369210 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:11:08.369229 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.369229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.369239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.369309 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.369613 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.370639 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.371495 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.371810 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.371877 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.376114 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.376115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.376116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.376123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.377738 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.378125 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.378126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.378126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.378181 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.378353 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.378354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.378355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.378496 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.378898 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.379250 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.380476 (+ 1226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260812 21:11:08.382287 4432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: Bootstrap starting.
W20260812 21:11:08.382340 4749 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:11:08.382382 4749 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:11:08.382933 4432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:08.383164 4432 log.cc:824] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:08.381803 4585 master_runner.cc:186] Error getting master registration for 127.11.207.190:39911: OK, instance_id { permanent_uuid: "970e33efcdd648048eb35e158bb32134" instance_seqno: 1786569068026476 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:11:08.383932 4432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: No bootstrap required, opened a new log
I20260812 21:11:08.383980 4585 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:11:08.387645 4432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.387802 4432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:08.387852 4432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970e33efcdd648048eb35e158bb32134, State: Initialized, Role: FOLLOWER
I20260812 21:11:08.388571 4432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.388949 4432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:08.390023 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.381026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.11.207.188:41405)
0812 21:11:08.381058 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.381060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.381321 (+ 261us) server_negotiation.cc:407] Connection header received
0812 21:11:08.381403 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.381404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.381413 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.381442 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.382140 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.382821 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.383786 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.384043 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.384089 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.385609 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.385611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.385611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.385617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.388843 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.389214 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.389216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.389216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.389267 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.389463 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.389464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.389465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.389580 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.389650 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.389863 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.389938 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260812 21:11:08.392843 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.383686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50014 (local address 127.11.207.190:39911)
0812 21:11:08.383726 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.383728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.384714 (+ 986us) server_negotiation.cc:407] Connection header received
0812 21:11:08.384809 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.384810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.384821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.384849 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.385404 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.386251 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.387424 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.387696 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.387749 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.389417 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.389418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.389418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.389425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.391637 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.392017 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.392018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.392018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.392084 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.392265 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.392266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.392266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.392407 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.392483 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.392674 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.392758 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:11:08.391623 4308 master_runner.cc:186] Error getting master registration for 127.11.207.189:41113: OK, instance_id { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" instance_seqno: 1786569068187902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:11:08.394861 4308 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:11:08.396178 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.386165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.11.207.189:41113)
0812 21:11:08.386209 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.386211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.390282 (+ 4071us) server_negotiation.cc:407] Connection header received
0812 21:11:08.390486 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.390487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.390497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.390523 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.391007 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.391697 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.392580 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.392839 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.392880 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.393953 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.393954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.393954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.393960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.395183 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.395469 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.395470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.395470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.395523 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.395723 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.395724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.395724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.395828 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.395904 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.396027 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.396093 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 21:11:08.397059 4778 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:11:08.397118 4778 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:11:08.397446 4760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.397526 4760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:11:08.421021 4573 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } attempt: 3
Aug 12 21:11:08 dist-test-slave-lkx2 krb5kdc[4275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569068, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:11:08.427448 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.421544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:41405 (local address 127.0.0.1:56614)
0812 21:11:08.421626 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:08.421636 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:08.421676 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:08.421857 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:08.421858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:08.422015 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:08.422057 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.422058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.422662 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.422664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.423170 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:08.423172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.423492 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.423493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:08.423545 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.423946 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:08.423950 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:08.425103 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:08.426515 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.426516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.426516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.426678 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.426860 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:08.426860 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:08.426861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:08.426917 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:08.427135 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:08.427136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:08.427182 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 21:11:08.427248 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:11:08.427348 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 21:11:08.427448 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:08.421589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.11.207.188:41405)
0812 21:11:08.421629 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:08.421631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:08.421737 (+ 106us) server_negotiation.cc:407] Connection header received
0812 21:11:08.421759 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:08.421760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:08.421769 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:08.421813 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:08.422134 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.422594 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.423256 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:08.423435 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:08.423488 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:08.425183 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:08.425184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:08.425185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:08.425192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:08.426436 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.426742 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.426743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.426743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.426792 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:08.426967 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:08.426968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:08.426968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:08.427078 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:08.427134 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:08.427278 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 21:11:08.427348 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:11:08.429009 4573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: Bootstrap starting.
I20260812 21:11:08.429724 4573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:08.430023 4573 log.cc:824] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:08.431005 4573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: No bootstrap required, opened a new log
I20260812 21:11:08.432629 4573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.432744 4573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:08.432777 4573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb3c79ef59d495f8e8d3cf5b33c4c16, State: Initialized, Role: FOLLOWER
I20260812 21:11:08.432904 4573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.433176 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.433241 4573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:08.433318 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:11:08.435154 4804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:11:08.435257 4804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 21:11:08.435679 4480 tablet.cc:2369] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:08.437394 4447 master_runner.cc:263] No leader master found from master 3cb3c79ef59d495f8e8d3cf5b33c4c16
I20260812 21:11:08.437443 4447 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 21:11:08.524412 4343 tablet.cc:2369] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:08.591285 4617 tablet.cc:2369] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:08.605325 4782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:08.605470 4782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.605758 4782 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 970e33efcdd648048eb35e158bb32134 (127.11.207.190:39911), 378387485b7f468aba661087ff530f05 (127.11.207.188:41405)
I20260812 21:11:08.606089 4378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970e33efcdd648048eb35e158bb32134" is_pre_election: true
I20260812 21:11:08.606240 4378 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cb3c79ef59d495f8e8d3cf5b33c4c16 in term 0.
I20260812 21:11:08.606149 4659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378387485b7f468aba661087ff530f05" is_pre_election: true
I20260812 21:11:08.606357 4659 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cb3c79ef59d495f8e8d3cf5b33c4c16 in term 0.
I20260812 21:11:08.606554 4476 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cb3c79ef59d495f8e8d3cf5b33c4c16, 970e33efcdd648048eb35e158bb32134; no voters:
I20260812 21:11:08.606726 4782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:11:08.606801 4782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:11:08.606817 4782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:08.607592 4782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.607715 4782 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [CANDIDATE]: Term 1 election: Requested vote from peers 970e33efcdd648048eb35e158bb32134 (127.11.207.190:39911), 378387485b7f468aba661087ff530f05 (127.11.207.188:41405)
I20260812 21:11:08.607916 4378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970e33efcdd648048eb35e158bb32134"
I20260812 21:11:08.607944 4659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378387485b7f468aba661087ff530f05"
I20260812 21:11:08.608017 4378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:08.608026 4659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:08.608973 4378 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cb3c79ef59d495f8e8d3cf5b33c4c16 in term 1.
I20260812 21:11:08.608973 4659 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cb3c79ef59d495f8e8d3cf5b33c4c16 in term 1.
I20260812 21:11:08.609185 4475 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378387485b7f468aba661087ff530f05, 3cb3c79ef59d495f8e8d3cf5b33c4c16; no voters:
I20260812 21:11:08.609319 4782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:08.609524 4782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [term 1 LEADER]: Becoming Leader. State: Replica: 3cb3c79ef59d495f8e8d3cf5b33c4c16, State: Running, Role: LEADER
I20260812 21:11:08.609628 4782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } }
I20260812 21:11:08.610042 4807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb3c79ef59d495f8e8d3cf5b33c4c16. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.610136 4807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:08.610379 4810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:08.610606 4810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:08.612080 4378 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [term 1 FOLLOWER]: Refusing update from remote peer 3cb3c79ef59d495f8e8d3cf5b33c4c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:08.612165 4659 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [term 1 FOLLOWER]: Refusing update from remote peer 3cb3c79ef59d495f8e8d3cf5b33c4c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:08.612411 4807 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:08.612537 4782 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:08.613824 4722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb3c79ef59d495f8e8d3cf5b33c4c16. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.613819 4760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb3c79ef59d495f8e8d3cf5b33c4c16. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.613909 4722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:11:08.613910 4760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:11:08.614917 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.615001 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:08.615067 4811 mvcc.cc:205] Tried to move back new op lower bound from 7317786905032593408 to 7317786905025548288. Current Snapshot: MvccSnapshot[applied={T|T < 7317786905032593408}]
I20260812 21:11:08.615041 4722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.615113 4722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:11:08.615172 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.615231 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16 [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:08.615450 4760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e33efcdd648048eb35e158bb32134" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 39911 } } peers { permanent_uuid: "3cb3c79ef59d495f8e8d3cf5b33c4c16" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 41113 } } peers { permanent_uuid: "378387485b7f468aba661087ff530f05" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 41405 } } }
I20260812 21:11:08.615530 4760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:11:08.615710 4810 catalog_manager.cc:1347] Generated new cluster ID: 6cfc59f6bf2c4351aaee1ea09b178f07
I20260812 21:11:08.615761 4810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:11:08.638047 4810 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:11:08.638636 4810 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:08.650851 4810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cb3c79ef59d495f8e8d3cf5b33c4c16: Generated new TSK 0
I20260812 21:11:08.651114 4810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:11:09.382910 4749 catalog_manager.cc:1259] Loaded cluster ID: 6cfc59f6bf2c4351aaee1ea09b178f07
I20260812 21:11:09.382993 4749 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: loading cluster ID for follower catalog manager: success
I20260812 21:11:09.383827 4749 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: acquiring CA information for follower catalog manager: success
I20260812 21:11:09.384122 4749 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 378387485b7f468aba661087ff530f05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:11:09.397580 4778 catalog_manager.cc:1259] Loaded cluster ID: 6cfc59f6bf2c4351aaee1ea09b178f07
I20260812 21:11:09.397647 4778 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: loading cluster ID for follower catalog manager: success
I20260812 21:11:09.398442 4778 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: acquiring CA information for follower catalog manager: success
I20260812 21:11:09.398713 4778 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 970e33efcdd648048eb35e158bb32134: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:11:09.705658 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 4308
I20260812 21:11:09.719777 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 4447
I20260812 21:11:09.739985 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 4585
2026-08-12T21:11:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3193 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:11:09 dist-test-slave-lkx2 krb5kdc[4836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:11:09 dist-test-slave-lkx2 krb5kdc[4836](info): set up 2 sockets
Aug 12 21:11:09 dist-test-slave-lkx2 krb5kdc[4836](info): commencing operation
krb5kdc: starting...
W20260812 21:11:10.965916 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:11:11 dist-test-slave-lkx2 krb5kdc[4836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:11:11.088649 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37021
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:37021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:11.171237 4869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:11.171451 4869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:11.171481 4869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:11.172665 4869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:11.172708 4869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:11.172726 4869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:11.172742 4869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:11.172758 4869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:11.174273 4869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:37021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:11.174515 4869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:11.174723 4869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:11.179450 4886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:11.179479 4879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:11.179605 4869 server_base.cc:1034] running on GCE node
W20260812 21:11:11.179450 4895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:11.179909 4869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:11.180214 4869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:11.181355 4869 hybrid_clock.cc:648] HybridClock initialized: now 1786569071181336 us; error 36 us; skew 500 ppm
Aug 12 21:11:11 dist-test-slave-lkx2 krb5kdc[4836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569071, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:11.183741 4869 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:11.184178 4869 webserver.cc:460] Webserver started at http://127.11.207.190:42011/ using document root <none> and password file <none>
I20260812 21:11:11.184428 4869 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:11.184481 4869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:11.184612 4869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:11.185631 4869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "e7333ee1041d4f26b8192ebdf7b1528a"
format_stamp: "Formatted at 2026-08-12 21:11:11 on dist-test-slave-lkx2"
server_key: "09c942daac8812af7766f9b84e599bf9"
server_key_iv: "4d3c2bc5f1c10ca0d9fad5acd03e8237"
server_key_version: "encryptionkey@0"
I20260812 21:11:11.185976 4869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "e7333ee1041d4f26b8192ebdf7b1528a"
format_stamp: "Formatted at 2026-08-12 21:11:11 on dist-test-slave-lkx2"
server_key: "09c942daac8812af7766f9b84e599bf9"
server_key_iv: "4d3c2bc5f1c10ca0d9fad5acd03e8237"
server_key_version: "encryptionkey@0"
I20260812 21:11:11.187261 4869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:11:11.188005 4908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:11.188159 4869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:11.188253 4869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "e7333ee1041d4f26b8192ebdf7b1528a"
format_stamp: "Formatted at 2026-08-12 21:11:11 on dist-test-slave-lkx2"
server_key: "09c942daac8812af7766f9b84e599bf9"
server_key_iv: "4d3c2bc5f1c10ca0d9fad5acd03e8237"
server_key_version: "encryptionkey@0"
I20260812 21:11:11.188328 4869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:11.214562 4869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:11.217561 4869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:11.217718 4869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:11.225916 4869 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:37021
I20260812 21:11:11.225956 5002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:37021 every 8 connection(s)
I20260812 21:11:11.226387 4869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:11.227280 5003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:11.229732 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a: Bootstrap starting.
I20260812 21:11:11.230327 5003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:11.230583 5003 log.cc:824] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:11.231238 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a: No bootstrap required, opened a new log
I20260812 21:11:11.232911 5003 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37021 } }
I20260812 21:11:11.233052 5003 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:11.233083 5003 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7333ee1041d4f26b8192ebdf7b1528a, State: Initialized, Role: FOLLOWER
I20260812 21:11:11.233222 5003 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37021 } }
I20260812 21:11:11.233305 5003 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:11.233347 5003 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:11.233407 5003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:11.233915 5003 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37021 } }
I20260812 21:11:11.234010 5003 leader_election.cc:302] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7333ee1041d4f26b8192ebdf7b1528a; no voters:
I20260812 21:11:11.234170 5003 leader_election.cc:288] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:11:11.234205 5013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:11.234347 5013 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [term 1 LEADER]: Becoming Leader. State: Replica: e7333ee1041d4f26b8192ebdf7b1528a, State: Running, Role: LEADER
I20260812 21:11:11.234407 5003 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:11.234440 5013 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37021 } }
I20260812 21:11:11.234664 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 4869
I20260812 21:11:11.234766 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:11:11.235008 12094 external_mini_cluster.cc:1428] Setting key 23e368f086a238855d4cd3926473b1d3
I20260812 21:11:11.235208 5015 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7333ee1041d4f26b8192ebdf7b1528a. Latest consensus state: current_term: 1 leader_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37021 } } }
I20260812 21:11:11.235284 5015 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:11.235394 5014 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7333ee1041d4f26b8192ebdf7b1528a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37021 } } }
I20260812 21:11:11.235479 5014 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a [sys.catalog]: This master's current role is: LEADER
W20260812 21:11:11.236394 5058 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:11:11.236444 5058 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:11:11.236487 5035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:11.236631 5035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:11.238710 5035 catalog_manager.cc:1347] Generated new cluster ID: c67fb26403094fc8bd49515d365b0a51
I20260812 21:11:11.238758 5035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:11:11 dist-test-slave-lkx2 krb5kdc[4836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:11.246289 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:11.235658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.11.207.190:37021)
0812 21:11:11.235822 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:11.235831 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:11.235915 (+ 84us) server_negotiation.cc:407] Connection header received
0812 21:11:11.236033 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:11.236040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:11.236095 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:11.236245 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:11.237001 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:11.238345 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:11.239418 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:11.239993 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:11.240088 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:11.242776 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:11.242781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:11.242782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:11.242794 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:11.244707 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:11.245062 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:11.245065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:11.245066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:11.245191 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:11.245379 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:11.245380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:11.245380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:11.245681 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:11.245839 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:11.245906 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 21:11:11.246044 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260812 21:11:11.247663 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:11:11.251171 5035 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:11:11.251641 5035 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:11.258023 5035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7333ee1041d4f26b8192ebdf7b1528a: Generated new TSK 0
I20260812 21:11:11.258174 5035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:11:11.301685 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 4869
2026-08-12T21:11:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1560 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:11:11 dist-test-slave-lkx2 krb5kdc[5067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:11:11 dist-test-slave-lkx2 krb5kdc[5067](info): set up 2 sockets
Aug 12 21:11:11 dist-test-slave-lkx2 krb5kdc[5067](info): commencing operation
krb5kdc: starting...
W20260812 21:11:12.501312 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:11:12 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:11:12.626940 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41269
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:12.710805 5092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:12.711038 5092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:12.711064 5092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:12.712249 5092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:12.712303 5092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:12.712319 5092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:12.712332 5092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:12.712370 5092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:12.713997 5092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:12.714249 5092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:12.714480 5092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:12.719563 5104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:12.719517 5107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:12.719517 5105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:12.719545 5092 server_base.cc:1034] running on GCE node
I20260812 21:11:12.720005 5092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:12.720273 5092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:12.721427 5092 hybrid_clock.cc:648] HybridClock initialized: now 1786569072721409 us; error 39 us; skew 500 ppm
Aug 12 21:11:12 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569072, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:12.729063 5092 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:12.729573 5092 webserver.cc:460] Webserver started at http://127.11.207.190:45469/ using document root <none> and password file <none>
I20260812 21:11:12.729784 5092 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:12.729826 5092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:12.729934 5092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:12.731074 5092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "f9fc3a8af1048618e6ac98c00308b97b"
server_key_iv: "8fb6308ee60d351342de7ac6b2831bd5"
server_key_version: "encryptionkey@0"
I20260812 21:11:12.731401 5092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "f9fc3a8af1048618e6ac98c00308b97b"
server_key_iv: "8fb6308ee60d351342de7ac6b2831bd5"
server_key_version: "encryptionkey@0"
I20260812 21:11:12.732752 5092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:11:12.733549 5116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:12.733758 5092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:11:12.733852 5092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "f9fc3a8af1048618e6ac98c00308b97b"
server_key_iv: "8fb6308ee60d351342de7ac6b2831bd5"
server_key_version: "encryptionkey@0"
I20260812 21:11:12.733922 5092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:12.749472 5092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:12.751514 5092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:12.751633 5092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:12.755800 5092 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41269
I20260812 21:11:12.755843 5221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41269 every 8 connection(s)
I20260812 21:11:12.756206 5092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:12.757264 5222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:12.759613 5222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap starting.
I20260812 21:11:12.760210 5222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:12.760528 5222 log.cc:824] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:12.761255 5222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: No bootstrap required, opened a new log
I20260812 21:11:12.762840 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 5092
I20260812 21:11:12.762956 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:11:12.763137 12094 external_mini_cluster.cc:1428] Setting key d3d610a0db2eac32cc86b2ea29229351
I20260812 21:11:12.763085 5222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:12.763274 5222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:12.763312 5222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f70339eecbb4ee9b29d8d1bf46b7e4a, State: Initialized, Role: FOLLOWER
I20260812 21:11:12.763458 5222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:12.763547 5222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:12.763579 5222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:12.763628 5222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:12.764791 5222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:12.764928 5222 leader_election.cc:302] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f70339eecbb4ee9b29d8d1bf46b7e4a; no voters:
I20260812 21:11:12.765101 5222 leader_election.cc:288] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:11:12.765143 5229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:12.765285 5229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 LEADER]: Becoming Leader. State: Replica: 1f70339eecbb4ee9b29d8d1bf46b7e4a, State: Running, Role: LEADER
I20260812 21:11:12.765385 5222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:12.765398 5229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:12.765799 5230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } } }
I20260812 21:11:12.765885 5230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:12.766038 5236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f70339eecbb4ee9b29d8d1bf46b7e4a. Latest consensus state: current_term: 1 leader_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } } }
I20260812 21:11:12.766093 5236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:12.766695 5248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:12.767381 5248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:12.768864 5248 catalog_manager.cc:1347] Generated new cluster ID: 0d582dfb024348ac8d5bc7fde1c17d52
I20260812 21:11:12.768923 5248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:11:12 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:12.773455 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:12.763643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44208 (local address 127.11.207.190:41269)
0812 21:11:12.763856 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:12.763865 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:12.763977 (+ 112us) server_negotiation.cc:407] Connection header received
0812 21:11:12.764130 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:12.764136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:12.764196 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:12.764309 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:12.765120 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:12.766509 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:12.767476 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:12.768078 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:12.768188 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:12.770490 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:12.770494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:12.770494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:12.770507 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:12.772027 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:12.772369 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:12.772372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:12.772373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:12.772472 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:12.772648 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:12.772649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:12.772650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:12.772880 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:12.773031 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:12.773108 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 21:11:12.773256 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260812 21:11:12.798116 5248 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:11:12.798695 5248 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:12.805250 5248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Generated new TSK 0
I20260812 21:11:12.805428 5248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:11:12.839325 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41269
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:12.947862 5274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:12.948076 5274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:12.948101 5274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:12.949265 5274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:12.949309 5274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:12.949334 5274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:12.950805 5274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:12.951026 5274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:12.951261 5274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:12.956180 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:12.956210 5281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:12.956190 5283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:12.956403 5274 server_base.cc:1034] running on GCE node
I20260812 21:11:12.956633 5274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:12.956851 5274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:12.958000 5274 hybrid_clock.cc:648] HybridClock initialized: now 1786569072957984 us; error 28 us; skew 500 ppm
Aug 12 21:11:12 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569072, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:12.960526 5274 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:12.960928 5274 webserver.cc:460] Webserver started at http://127.11.207.129:46647/ using document root <none> and password file <none>
I20260812 21:11:12.961110 5274 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:12.961148 5274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:12.961249 5274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:12.962314 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "72a4b9e6e19b489b93102bd04315ab23"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "af6e5e7e9a5264d6a828027bcecc259e"
server_key_iv: "7327581df8d3c824b40e5826a96060eb"
server_key_version: "encryptionkey@0"
I20260812 21:11:12.962615 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "72a4b9e6e19b489b93102bd04315ab23"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "af6e5e7e9a5264d6a828027bcecc259e"
server_key_iv: "7327581df8d3c824b40e5826a96060eb"
server_key_version: "encryptionkey@0"
I20260812 21:11:12.963932 5274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:11:12.964694 5303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:12.964860 5274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:12.964947 5274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "72a4b9e6e19b489b93102bd04315ab23"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "af6e5e7e9a5264d6a828027bcecc259e"
server_key_iv: "7327581df8d3c824b40e5826a96060eb"
server_key_version: "encryptionkey@0"
I20260812 21:11:12.965006 5274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:12.972450 5274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:12.974303 5274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:12.974416 5274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:12.974608 5274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:12.974924 5274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:12.974956 5274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:12.974977 5274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:12.974988 5274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:12.980816 5274 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:33795
I20260812 21:11:12.980906 5504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:33795 every 8 connection(s)
I20260812 21:11:12.981180 5274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:11:12.986368 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 5274
I20260812 21:11:12.986487 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:11:12.986723 12094 external_mini_cluster.cc:1428] Setting key 85447454b0784efc82022851e4e60fb4
Aug 12 21:11:12 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569072, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:12.991011 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:12.981900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:34653 (local address 127.11.207.190:41269)
0812 21:11:12.981982 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:12.981984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:12.982164 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:11:12.982656 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:12.982658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:12.982672 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:12.982757 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:12.983738 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:12.984242 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:12.985260 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:12.985465 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:12.985523 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:12.987811 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:12.987813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:12.987814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:12.987823 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:12.989366 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:12.989904 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:12.989905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:12.989905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:12.989970 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:12.990265 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:12.990266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:12.990266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:12.990392 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:12.990456 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:12.990823 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 21:11:12.990922 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 21:11:12.991096 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:12.981870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:34653)
0812 21:11:12.982046 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:12.982084 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:12.982573 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:12.982826 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:12.982830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:12.983425 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:12.983631 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:12.983636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:12.984311 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:12.984312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:12.985174 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:12.985178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:12.985529 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:12.985529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:12.985625 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:12.986129 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:12.986138 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:12.987710 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:12.989457 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:12.989468 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:12.989471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:12.989773 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:12.990030 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:12.990031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:12.990031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:12.990176 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:12.990459 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:12.990467 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:12.990643 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 21:11:12.990753 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 21:11:12.990915 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260812 21:11:12.991639 5505 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41269
I20260812 21:11:12.991741 5505 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:12.991945 5505 heartbeater.cc:511] Master 127.11.207.190:41269 requested a full tablet report, sending...
I20260812 21:11:12.992607 5153 ts_manager.cc:194] Registered new tserver with Master: 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795)
I20260812 21:11:12.993564 5153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:34653
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:11:13.015549 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41269
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:13.101536 5519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:13.101774 5519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:13.101800 5519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:13.102955 5519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:13.102993 5519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:13.103019 5519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:13.104524 5519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:13.104754 5519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:13.104943 5519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:13.110404 5528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:13.110390 5527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:13.110389 5531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:13.110673 5519 server_base.cc:1034] running on GCE node
I20260812 21:11:13.110932 5519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:13.111207 5519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:13.112382 5519 hybrid_clock.cc:648] HybridClock initialized: now 1786569073112363 us; error 34 us; skew 500 ppm
Aug 12 21:11:13 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569073, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:13.115145 5519 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:13.115607 5519 webserver.cc:460] Webserver started at http://127.11.207.130:45671/ using document root <none> and password file <none>
I20260812 21:11:13.115857 5519 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:13.115911 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:13.116040 5519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:13.117188 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "ce9eddc553d020c5d7fb6ab18231f8cd"
server_key_iv: "8330c695ef20c582ded9751dd3e88879"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.117539 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "ce9eddc553d020c5d7fb6ab18231f8cd"
server_key_iv: "8330c695ef20c582ded9751dd3e88879"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.118921 5519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:11:13.119735 5548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.119941 5519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:13.120043 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "ce9eddc553d020c5d7fb6ab18231f8cd"
server_key_iv: "8330c695ef20c582ded9751dd3e88879"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.120110 5519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:13.130496 5519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:13.132628 5519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:13.132794 5519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:13.133035 5519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:13.133373 5519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:13.133423 5519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.133455 5519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:13.133476 5519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.139319 5519 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:37153
I20260812 21:11:13.139392 5765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:37153 every 8 connection(s)
I20260812 21:11:13.139685 5519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:13.141765 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 5519
I20260812 21:11:13.141881 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:11:13.142095 12094 external_mini_cluster.cc:1428] Setting key e4b4f7ef79fa0aeffdd1409ba81bd2e7
Aug 12 21:11:13 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569073, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:13.150166 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.140450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:48623 (local address 127.11.207.190:41269)
0812 21:11:13.140520 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.140521 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.140743 (+ 222us) server_negotiation.cc:407] Connection header received
0812 21:11:13.141275 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.141277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.141288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.141349 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:13.142478 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.142977 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.144408 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.144678 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.144779 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.147086 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:13.147088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:13.147088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:13.147096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:13.148527 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:13.149068 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:13.149069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:13.149069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:13.149137 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:13.149469 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:13.149469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:13.149470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:13.149581 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:13.149631 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.150011 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.150096 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:11:13.150434 5774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.140461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:48623)
0812 21:11:13.140622 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.140659 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.141161 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.141420 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.141424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.142095 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:13.142365 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.142371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.143068 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.143070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.144251 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.144255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.144785 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.144786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.144910 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.145422 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:13.145431 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:13.146948 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:13.148626 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:13.148643 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:13.148646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:13.148981 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:13.149204 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:13.149205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:13.149206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:13.149376 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:13.149647 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:13.149653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:13.149839 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.149973 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.150170 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260812 21:11:13.151022 5768 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41269
I20260812 21:11:13.151131 5768 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:13.151327 5768 heartbeater.cc:511] Master 127.11.207.190:41269 requested a full tablet report, sending...
I20260812 21:11:13.151752 5153 ts_manager.cc:194] Registered new tserver with Master: 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153)
I20260812 21:11:13.152153 5153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:48623
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:11:13.169317 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41269
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:13.251430 5779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:13.251642 5779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:13.251675 5779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:13.252849 5779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:13.252890 5779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:13.252926 5779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:13.254408 5779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:13.254649 5779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:13.254873 5779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:13.259722 5793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:13.259745 5795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:13.259760 5792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:13.259868 5779 server_base.cc:1034] running on GCE node
I20260812 21:11:13.260143 5779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:13.260423 5779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:13.261564 5779 hybrid_clock.cc:648] HybridClock initialized: now 1786569073261545 us; error 37 us; skew 500 ppm
Aug 12 21:11:13 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569073, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:13.263968 5779 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:13.264494 5779 webserver.cc:460] Webserver started at http://127.11.207.131:33457/ using document root <none> and password file <none>
I20260812 21:11:13.264750 5779 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:13.264810 5779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:13.264938 5779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:13.266031 5779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "763bc8b9ba6c47879c037aca28c69864"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "4a820ce7c2d29ff1b670985bed1fed97"
server_key_iv: "f056964b257c13c568919fd15d1c70b3"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.266407 5779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "763bc8b9ba6c47879c037aca28c69864"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "4a820ce7c2d29ff1b670985bed1fed97"
server_key_iv: "f056964b257c13c568919fd15d1c70b3"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.267765 5779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:11:13.268524 5804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.268692 5779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:13.268785 5779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "763bc8b9ba6c47879c037aca28c69864"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "4a820ce7c2d29ff1b670985bed1fed97"
server_key_iv: "f056964b257c13c568919fd15d1c70b3"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.268862 5779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:13.308758 5779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:13.311020 5779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:13.311137 5779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:13.311352 5779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:13.311684 5779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:13.311718 5779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.311743 5779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:13.311766 5779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.317633 5779 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:33495
I20260812 21:11:13.317688 6032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:33495 every 8 connection(s)
I20260812 21:11:13.318086 5779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:11:13 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569073, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:13.325093 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 5779
I20260812 21:11:13.325192 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:11:13.325359 12094 external_mini_cluster.cc:1428] Setting key 60a826cde8f8b5db9c5ab271c735c7bd
I20260812 21:11:13.327347 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.318855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:55879 (local address 127.11.207.190:41269)
0812 21:11:13.318912 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.318914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.319085 (+ 171us) server_negotiation.cc:407] Connection header received
0812 21:11:13.319591 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.319593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.319604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.319672 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:13.320676 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.321173 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.322285 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.322475 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.322528 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.324760 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:13.324762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:13.324763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:13.324771 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:13.326247 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:13.326606 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:13.326607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:13.326607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:13.326658 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:13.326860 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:13.326860 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:13.326861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:13.326962 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:13.327010 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.327220 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.327281 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:11:13.327543 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.318799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:55879)
0812 21:11:13.318961 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.319004 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.319492 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.319727 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.319731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.320317 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:13.320579 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.320585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.321246 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.321246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.322191 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.322195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.322546 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.322547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.322656 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.323220 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:13.323230 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:13.324674 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:13.326315 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:13.326324 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:13.326327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:13.326549 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:13.326702 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:13.326703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:13.326703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:13.326797 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:13.327005 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:13.327009 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:13.327125 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.327203 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.327327 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260812 21:11:13.327981 6034 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41269
I20260812 21:11:13.328078 6034 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:13.328271 6034 heartbeater.cc:511] Master 127.11.207.190:41269 requested a full tablet report, sending...
I20260812 21:11:13.328644 5153 ts_manager.cc:194] Registered new tserver with Master: 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
I20260812 21:11:13.329018 5153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:55879
I20260812 21:11:13.337241 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:11:13.344308 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.338591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.11.207.190:41269)
0812 21:11:13.338658 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.338660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.338741 (+ 81us) server_negotiation.cc:407] Connection header received
0812 21:11:13.338811 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.338812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.338823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.338873 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:13.339443 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.339935 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.340704 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.340901 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.340954 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.342062 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:13.342064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:13.342064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:13.342095 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:13.343348 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:13.343674 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:13.343675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:13.343675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:13.343722 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:13.343877 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:13.343878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:13.343878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:13.344024 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:13.344099 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.344174 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.344240 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:11:13.345865 5153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:11:13.347374 5153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:11:13.353869 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.349928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.11.207.130:37153)
0812 21:11:13.350129 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.350135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.350169 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:11:13.350226 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.350228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.350243 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.350322 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:13.350576 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.351344 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.352809 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.353574 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.353647 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.353700 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.353762 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.353802 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
I20260812 21:11:13.353852 6057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.349781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:37153 (local address 127.0.0.1:40098)
0812 21:11:13.350123 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.350130 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.350167 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.350428 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.350429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.350432 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:13.350489 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.350491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.351436 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.351437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.352719 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.352722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.353644 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.353645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.353700 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.353703 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.353750 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.353786 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":79,"threads_started":1}
I20260812 21:11:13.354032 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.349742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.11.207.129:33795)
0812 21:11:13.350154 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.350159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.350168 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:11:13.350201 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.350203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.350215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.350273 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:13.350467 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.351589 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.352847 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.353425 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.353496 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.353558 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.353902 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.353948 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":129,"threads_started":1}
I20260812 21:11:13.353850 6054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.349618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33795 (local address 127.0.0.1:56848)
0812 21:11:13.349970 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.349979 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.350030 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.350300 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.350303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.350305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:13.350379 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.350381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.351690 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.351691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.352763 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.352765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.353503 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.353504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.353596 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.353611 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.353680 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.353733 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260812 21:11:13.354481 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.349590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33495 (local address 127.0.0.1:47560)
0812 21:11:13.349908 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.349953 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.350027 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.350642 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.350643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.350645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:13.350680 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.350681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.351878 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.351879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.352744 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.352747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.354304 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.354305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.354362 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.354366 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.354407 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.354436 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":58,"threads_started":1}
I20260812 21:11:13.354557 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.349710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.11.207.131:33495)
0812 21:11:13.350434 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.350437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.350446 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:11:13.350483 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.350484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.350499 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.350568 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:13.350763 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.351792 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.352839 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.353573 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.353647 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.353700 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.354449 (+ 749us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.354492 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":81,"threads_started":1}
I20260812 21:11:13.355254 5641 tablet_service.cc:1467] Processing CreateTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 (DEFAULT_TABLE table=test-table [id=57f2e8b8bdb94dc4b52afcc687182369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:13.355252 5910 tablet_service.cc:1467] Processing CreateTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 (DEFAULT_TABLE table=test-table [id=57f2e8b8bdb94dc4b52afcc687182369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:13.355566 5641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30eb1ac4142c438fb0ed6f4d45d15574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:13.355568 5910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30eb1ac4142c438fb0ed6f4d45d15574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:13.358145 6060 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Bootstrap starting.
I20260812 21:11:13.358366 6062 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Bootstrap starting.
I20260812 21:11:13.358673 5386 tablet_service.cc:1467] Processing CreateTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 (DEFAULT_TABLE table=test-table [id=57f2e8b8bdb94dc4b52afcc687182369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:13.358879 6060 tablet_bootstrap.cc:654] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:13.358958 6062 tablet_bootstrap.cc:654] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:13.358984 5386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30eb1ac4142c438fb0ed6f4d45d15574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:13.359457 6060 log.cc:824] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:13.359531 6062 log.cc:824] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:13.360205 6060 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: No bootstrap required, opened a new log
I20260812 21:11:13.360263 6060 ts_tablet_manager.cc:1397] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:13.360996 6062 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: No bootstrap required, opened a new log
I20260812 21:11:13.361068 6062 ts_tablet_manager.cc:1397] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 21:11:13.361908 6067 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap starting.
I20260812 21:11:13.362174 6060 raft_consensus.cc:348] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.362350 6060 raft_consensus.cc:374] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:13.362389 6060 raft_consensus.cc:729] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763bc8b9ba6c47879c037aca28c69864, State: Initialized, Role: FOLLOWER
I20260812 21:11:13.362540 6067 tablet_bootstrap.cc:654] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:13.362530 6060 consensus_queue.cc:260] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.362820 6060 ts_tablet_manager.cc:1428] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:11:13.362838 6034 heartbeater.cc:503] Master 127.11.207.190:41269 was elected leader, sending a full tablet report...
I20260812 21:11:13.363051 6062 raft_consensus.cc:348] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.363124 6067 log.cc:824] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:13.363178 6062 raft_consensus.cc:374] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:13.363210 6062 raft_consensus.cc:729] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bccc4bfaa684ea2a28ce833f909fd7e, State: Initialized, Role: FOLLOWER
I20260812 21:11:13.363346 6062 consensus_queue.cc:260] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.363634 5768 heartbeater.cc:503] Master 127.11.207.190:41269 was elected leader, sending a full tablet report...
I20260812 21:11:13.363619 6062 ts_tablet_manager.cc:1428] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:11:13.363848 6067 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: No bootstrap required, opened a new log
I20260812 21:11:13.363912 6067 ts_tablet_manager.cc:1397] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:11:13.365561 6067 raft_consensus.cc:348] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.365711 6067 raft_consensus.cc:374] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:13.365751 6067 raft_consensus.cc:729] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a4b9e6e19b489b93102bd04315ab23, State: Initialized, Role: FOLLOWER
I20260812 21:11:13.365859 6067 consensus_queue.cc:260] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.366106 6067 ts_tablet_manager.cc:1428] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:13.366189 5505 heartbeater.cc:503] Master 127.11.207.190:41269 was elected leader, sending a full tablet report...
W20260812 21:11:13.390255 5769 tablet.cc:2369] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:13.481963 5510 tablet.cc:2369] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:13.568663 6037 tablet.cc:2369] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:13.624544 6070 raft_consensus.cc:482] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:13.624692 6070 raft_consensus.cc:504] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.625140 6070 leader_election.cc:288] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795)
I20260812 21:11:13.629077 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.625459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:52479 (local address 127.11.207.129:33795)
0812 21:11:13.625529 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.625531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.625704 (+ 173us) server_negotiation.cc:407] Connection header received
0812 21:11:13.625738 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.625739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.625751 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.625794 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:13.625925 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.626632 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.628109 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.628720 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.628807 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.628818 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.628960 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.629003 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:11:13.629077 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.625494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:33157 (local address 127.11.207.130:37153)
0812 21:11:13.625574 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.625577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.625587 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:11:13.625618 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.625619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.625631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.625685 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:13.625883 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.626632 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.628108 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.628709 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.628808 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.628817 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.628960 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.628999 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:11:13.629105 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.625366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:37153 (local address 127.11.207.131:33157)
0812 21:11:13.625430 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.625444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.625515 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.625730 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.625731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.625732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:13.625816 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.625819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.626714 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.626715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.628007 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.628011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.628794 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.628795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.628884 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.628890 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.628949 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.629002 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":49}
I20260812 21:11:13.629112 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.625377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33795 (local address 127.11.207.131:52479)
0812 21:11:13.625622 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.625630 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:13.625685 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:13.625805 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:13.625806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:13.625807 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:13.625864 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.625865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.626749 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.626750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.628007 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:13.628011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.628839 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.628840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:13.628895 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.628898 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:11:13.628949 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:11:13.629001 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":95,"thread_start_us":125,"threads_started":1}
I20260812 21:11:13.629515 5681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb1ac4142c438fb0ed6f4d45d15574" candidate_uuid: "763bc8b9ba6c47879c037aca28c69864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" is_pre_election: true
I20260812 21:11:13.629530 5427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb1ac4142c438fb0ed6f4d45d15574" candidate_uuid: "763bc8b9ba6c47879c037aca28c69864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a4b9e6e19b489b93102bd04315ab23" is_pre_election: true
I20260812 21:11:13.629662 5681 raft_consensus.cc:2454] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763bc8b9ba6c47879c037aca28c69864 in term 0.
I20260812 21:11:13.629676 5427 raft_consensus.cc:2454] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763bc8b9ba6c47879c037aca28c69864 in term 0.
I20260812 21:11:13.629913 5805 leader_election.cc:302] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23, 763bc8b9ba6c47879c037aca28c69864; no voters:
I20260812 21:11:13.630055 6070 raft_consensus.cc:2781] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:11:13.630113 6070 raft_consensus.cc:482] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:11:13.630129 6070 raft_consensus.cc:3037] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:13.631055 6070 raft_consensus.cc:504] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.631219 6070 leader_election.cc:288] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [CANDIDATE]: Term 1 election: Requested vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795)
I20260812 21:11:13.631409 5681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb1ac4142c438fb0ed6f4d45d15574" candidate_uuid: "763bc8b9ba6c47879c037aca28c69864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
I20260812 21:11:13.631425 5427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30eb1ac4142c438fb0ed6f4d45d15574" candidate_uuid: "763bc8b9ba6c47879c037aca28c69864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72a4b9e6e19b489b93102bd04315ab23"
I20260812 21:11:13.631500 5681 raft_consensus.cc:3037] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:13.631500 5427 raft_consensus.cc:3037] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:13.632480 5681 raft_consensus.cc:2454] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763bc8b9ba6c47879c037aca28c69864 in term 1.
I20260812 21:11:13.632480 5427 raft_consensus.cc:2454] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763bc8b9ba6c47879c037aca28c69864 in term 1.
I20260812 21:11:13.632638 5805 leader_election.cc:302] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23, 763bc8b9ba6c47879c037aca28c69864; no voters:
I20260812 21:11:13.632747 6070 raft_consensus.cc:2781] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:13.632911 6070 raft_consensus.cc:686] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 1 LEADER]: Becoming Leader. State: Replica: 763bc8b9ba6c47879c037aca28c69864, State: Running, Role: LEADER
I20260812 21:11:13.633014 6070 consensus_queue.cc:237] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:13.633847 5150 catalog_manager.cc:5495] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 reported cstate change: term changed from 0 to 1, leader changed from <none> to 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131). New cstate: current_term: 1 leader_uuid: "763bc8b9ba6c47879c037aca28c69864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } health_report { overall_health: UNKNOWN } } }
I20260812 21:11:13.705793 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.703560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.11.207.131:33495)
0812 21:11:13.703625 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:13.703627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:13.703660 (+ 33us) server_negotiation.cc:407] Connection header received
0812 21:11:13.703712 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:13.703713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:13.703732 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:13.703789 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:13.703923 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.704422 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.705122 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:13.705316 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:13.705376 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:13.705484 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:13.705592 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:11:13.705648 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:13.705665 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 21:11:13.705689 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:11:13.707718 5427 raft_consensus.cc:1261] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Refusing update from remote peer 763bc8b9ba6c47879c037aca28c69864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:13.707718 5681 raft_consensus.cc:1261] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Refusing update from remote peer 763bc8b9ba6c47879c037aca28c69864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:13.708086 6070 consensus_queue.cc:1035] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:13.708243 6079 consensus_queue.cc:1035] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:13.711778 6088 mvcc.cc:205] Tried to move back new op lower bound from 7317786925903892480 to 7317786925601378304. Current Snapshot: MvccSnapshot[applied={T|T < 7317786925903892480}]
I20260812 21:11:13.713203 5150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:13.713303 5150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:13.714061 5150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Sending DeleteTablet for 3 replicas of tablet 30eb1ac4142c438fb0ed6f4d45d15574
I20260812 21:11:13.714373 5910 tablet_service.cc:1514] Processing DeleteTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:13 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:47560
I20260812 21:11:13.714407 5386 tablet_service.cc:1514] Processing DeleteTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:13 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:56848
I20260812 21:11:13.714430 5641 tablet_service.cc:1514] Processing DeleteTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:13 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:40098
I20260812 21:11:13.714637 6107 tablet_replica.cc:330] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: stopping tablet replica
I20260812 21:11:13.714725 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 5274
I20260812 21:11:13.714756 6107 raft_consensus.cc:2229] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:13.722954 6104 tablet_replica.cc:330] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: stopping tablet replica
I20260812 21:11:13.723073 6104 raft_consensus.cc:2229] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:11:13.723165 6104 raft_consensus.cc:2258] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:13.726809 6107 raft_consensus.cc:2258] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:13.726886 6104 ts_tablet_manager.cc:1905] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:13.727279 6107 ts_tablet_manager.cc:1905] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:13.728618 6104 ts_tablet_manager.cc:1918] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:13.728691 6104 log.cc:1196] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/30eb1ac4142c438fb0ed6f4d45d15574
I20260812 21:11:13.728866 6107 ts_tablet_manager.cc:1918] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:13.728935 6107 log.cc:1196] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/30eb1ac4142c438fb0ed6f4d45d15574
I20260812 21:11:13.728965 6104 ts_tablet_manager.cc:1939] T 30eb1ac4142c438fb0ed6f4d45d15574 P 763bc8b9ba6c47879c037aca28c69864: Deleting consensus metadata
I20260812 21:11:13.729157 6107 ts_tablet_manager.cc:1939] T 30eb1ac4142c438fb0ed6f4d45d15574 P 6bccc4bfaa684ea2a28ce833f909fd7e: Deleting consensus metadata
I20260812 21:11:13.729437 5120 catalog_manager.cc:4841] TS 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): tablet 30eb1ac4142c438fb0ed6f4d45d15574 (table test-table [id=57f2e8b8bdb94dc4b52afcc687182369]) successfully deleted
I20260812 21:11:13.729586 5120 catalog_manager.cc:4841] TS 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): tablet 30eb1ac4142c438fb0ed6f4d45d15574 (table test-table [id=57f2e8b8bdb94dc4b52afcc687182369]) successfully deleted
W20260812 21:11:13.736836 5117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:33795 (error 108)
I20260812 21:11:13.737437 6054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:13.737305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33795 (local address 127.0.0.1:56862)
0812 21:11:13.737357 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:13.737399 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:11:13.737501 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 5519
W20260812 21:11:13.737571 5117 catalog_manager.cc:4584] TS 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30eb1ac4142c438fb0ed6f4d45d15574: Network error: Client connection negotiation failed: client connection to 127.11.207.129:33795: connect: Connection refused (error 111)
I20260812 21:11:13.748757 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 5779
I20260812 21:11:13.762660 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 5092
I20260812 21:11:13.773967 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41269
--webserver_interface=127.11.207.190
--webserver_port=45469
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:13.858662 6110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:13.858897 6110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:13.858928 6110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:13.860106 6110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:13.860149 6110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:13.860168 6110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:13.860183 6110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:13.860200 6110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:13.861789 6110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:13.862046 6110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:13.862260 6110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:11:13.872610 6110 server_base.cc:1034] running on GCE node
W20260812 21:11:13.872576 6126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:13.872592 6127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:13.872572 6130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:13.873080 6110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:13.873409 6110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:13.874559 6110 hybrid_clock.cc:648] HybridClock initialized: now 1786569073874547 us; error 29 us; skew 500 ppm
Aug 12 21:11:13 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569073, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:13.876992 6110 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:13.877466 6110 webserver.cc:460] Webserver started at http://127.11.207.190:45469/ using document root <none> and password file <none>
I20260812 21:11:13.877686 6110 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:13.877753 6110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:13.879132 6110 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:13.879793 6146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.879963 6110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:13.880053 6110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "f9fc3a8af1048618e6ac98c00308b97b"
server_key_iv: "8fb6308ee60d351342de7ac6b2831bd5"
server_key_version: "encryptionkey@0"
I20260812 21:11:13.880496 6110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:13.897521 6110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:13.899475 6110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:13.899619 6110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:13.903743 6110 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41269
I20260812 21:11:13.903790 6234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41269 every 8 connection(s)
I20260812 21:11:13.904181 6110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:13.905910 6236 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:11:13.906611 6236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap starting.
I20260812 21:11:13.909305 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 6110
I20260812 21:11:13.909585 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:33795
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:41269
--webserver_port=46647
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:13.916224 6236 log.cc:824] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:13.918226 6236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:13.918524 6236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap complete.
I20260812 21:11:13.920873 6236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:13.921114 6236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f70339eecbb4ee9b29d8d1bf46b7e4a, State: Initialized, Role: FOLLOWER
I20260812 21:11:13.921281 6236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:13.921361 6236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:13.921417 6236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:13.921466 6236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:11:13.922498 6236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:13.922597 6236 leader_election.cc:302] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f70339eecbb4ee9b29d8d1bf46b7e4a; no voters:
I20260812 21:11:13.922767 6236 leader_election.cc:288] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:11:13.922840 6243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:11:13.922999 6243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 LEADER]: Becoming Leader. State: Replica: 1f70339eecbb4ee9b29d8d1bf46b7e4a, State: Running, Role: LEADER
I20260812 21:11:13.923070 6236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:13.923102 6243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:13.923375 6243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f70339eecbb4ee9b29d8d1bf46b7e4a. Latest consensus state: current_term: 2 leader_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } } }
I20260812 21:11:13.923444 6243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:13.923580 6244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } } }
I20260812 21:11:13.923630 6244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:13.924041 6251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:13.924889 6251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:13.924990 6251 catalog_manager.cc:1259] Loaded cluster ID: 0d582dfb024348ac8d5bc7fde1c17d52
I20260812 21:11:13.925024 6251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:11:13.925501 6251 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 21:11:13.930505 6251 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 21:11:13.933143 6251 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:13.933363 6251 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Loaded TSK: 0
I20260812 21:11:13.933593 6251 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:11:13.997618 6238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:13.997828 6238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:13.997850 6238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:13.999074 6238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:13.999116 6238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:13.999141 6238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:14.000672 6238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:33795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=46647
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:14.000892 6238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:14.001117 6238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:14.007055 6287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:14.007063 6289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:14.007316 6292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:14.007448 6238 server_base.cc:1034] running on GCE node
I20260812 21:11:14.007659 6238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:14.007942 6238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:14.009120 6238 hybrid_clock.cc:648] HybridClock initialized: now 1786569074009098 us; error 34 us; skew 500 ppm
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:14.012141 6238 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:14.012638 6238 webserver.cc:460] Webserver started at http://127.11.207.129:46647/ using document root <none> and password file <none>
I20260812 21:11:14.012878 6238 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:14.012944 6238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:14.014351 6238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:14.015043 6310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.015220 6238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:14.015298 6238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "72a4b9e6e19b489b93102bd04315ab23"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "af6e5e7e9a5264d6a828027bcecc259e"
server_key_iv: "7327581df8d3c824b40e5826a96060eb"
server_key_version: "encryptionkey@0"
I20260812 21:11:14.015759 6238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:14.044941 6238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:14.047353 6238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:14.047503 6238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:14.047792 6238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:14.048321 6319 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:11:14.049836 6238 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:11:14.049896 6238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:11:14.049929 6238 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:11:14.050874 6238 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:11:14.050920 6238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:14.050982 6319 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap starting.
I20260812 21:11:14.058346 6238 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:33795
I20260812 21:11:14.058624 6521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:33795 every 8 connection(s)
I20260812 21:11:14.058773 6238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:11:14.061980 6319 log.cc:824] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:14.064374 6319 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:14.064669 6319 tablet_bootstrap.cc:492] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap complete.
I20260812 21:11:14.065788 6319 ts_tablet_manager.cc:1397] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260812 21:11:14.065918 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 6238
I20260812 21:11:14.066205 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:37153
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:41269
--webserver_port=45671
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:14.067059 6319 raft_consensus.cc:348] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.067373 6319 raft_consensus.cc:729] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a4b9e6e19b489b93102bd04315ab23, State: Initialized, Role: FOLLOWER
I20260812 21:11:14.067505 6319 consensus_queue.cc:260] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.067761 6319 ts_tablet_manager.cc:1428] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 21:11:14.071161 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.059607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:42897)
0812 21:11:14.060463 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.060529 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.061211 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.061654 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.061659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.062047 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:14.062316 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.062322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.063444 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.063446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.064722 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.064725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.065257 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.065258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.065410 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.065967 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:14.065978 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:14.067388 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:14.069577 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.069585 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.069587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.069813 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.070045 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.070046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.070046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.070146 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.070514 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:14.070521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:14.070672 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.070777 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.070964 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":91,"threads_started":1}
I20260812 21:11:14.071161 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.059704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:42897 (local address 127.11.207.190:41269)
0812 21:11:14.059936 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.059950 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.060663 (+ 713us) server_negotiation.cc:407] Connection header received
0812 21:11:14.061427 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.061433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.061476 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.061584 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:14.062439 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.063332 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.064801 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.065178 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.065256 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.067528 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:14.067532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:14.067532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:14.067544 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:14.069509 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.069888 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.069891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.069891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.069988 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.070201 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.070202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.070202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.070449 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:14.070524 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.070844 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.070967 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260812 21:11:14.071971 6522 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41269
I20260812 21:11:14.072063 6522 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:14.072235 6522 heartbeater.cc:511] Master 127.11.207.190:41269 requested a full tablet report, sending...
I20260812 21:11:14.072994 6172 ts_manager.cc:194] Registered new tserver with Master: 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795)
I20260812 21:11:14.074043 6172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:42897
I20260812 21:11:14.074887 6522 heartbeater.cc:503] Master 127.11.207.190:41269 was elected leader, sending a full tablet report...
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569073, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.129@KRBTEST.COM
I20260812 21:11:14.081640 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.073815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.11.207.129:33795)
0812 21:11:14.073981 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.073985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.073995 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:11:14.074029 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.074031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.074047 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.074221 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:14.074742 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.075625 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.076703 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.076981 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.077032 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.079141 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:14.079148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:14.079149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:14.079155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:14.080532 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.080944 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.080946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.080947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.080997 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.081164 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.081164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.081164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.081277 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:14.081342 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.081506 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.081577 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260812 21:11:14.081732 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.073564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33795 (local address 127.0.0.1:56870)
0812 21:11:14.073753 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.073802 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.073874 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.074232 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.074239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.074590 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:14.074656 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.074659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.075699 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.075701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.076623 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.076627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.077052 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.077053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.077138 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.077569 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:14.077575 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:14.079066 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:14.080601 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.080608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.080610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.080867 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.081048 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.081048 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.081049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.081113 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.081335 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:14.081341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:14.081399 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.081484 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.081602 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260812 21:11:14.082139 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet 30eb1ac4142c438fb0ed6f4d45d15574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:13 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:56870
I20260812 21:11:14.082762 6535 tablet_replica.cc:330] stopping tablet replica
I20260812 21:11:14.082868 6535 raft_consensus.cc:2229] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:14.082939 6535 raft_consensus.cc:2258] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:14.083264 6535 ts_tablet_manager.cc:1905] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:14.085608 6535 ts_tablet_manager.cc:1918] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:14.085690 6535 log.cc:1196] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/30eb1ac4142c438fb0ed6f4d45d15574
I20260812 21:11:14.085896 6535 ts_tablet_manager.cc:1939] T 30eb1ac4142c438fb0ed6f4d45d15574 P 72a4b9e6e19b489b93102bd04315ab23: Deleting consensus metadata
I20260812 21:11:14.086371 6147 catalog_manager.cc:4841] TS 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795): tablet 30eb1ac4142c438fb0ed6f4d45d15574 (table test-table [id=57f2e8b8bdb94dc4b52afcc687182369]) successfully deleted
W20260812 21:11:14.153219 6530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:14.153450 6530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:14.153475 6530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:14.154659 6530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:14.154701 6530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:14.154727 6530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:14.156196 6530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:37153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=45671
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:14.156443 6530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:14.156668 6530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:14.161597 6548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:14.161621 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:14.161733 6530 server_base.cc:1034] running on GCE node
W20260812 21:11:14.161903 6546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:14.162137 6530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:14.162384 6530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:14.163528 6530 hybrid_clock.cc:648] HybridClock initialized: now 1786569074163514 us; error 29 us; skew 500 ppm
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:14.165972 6530 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:14.166419 6530 webserver.cc:460] Webserver started at http://127.11.207.130:45671/ using document root <none> and password file <none>
I20260812 21:11:14.166616 6530 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:14.166678 6530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:14.167955 6530 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:14.168532 6565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.168680 6530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.168767 6530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "ce9eddc553d020c5d7fb6ab18231f8cd"
server_key_iv: "8330c695ef20c582ded9751dd3e88879"
server_key_version: "encryptionkey@0"
I20260812 21:11:14.169106 6530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:14.181037 6530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:14.183197 6530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:14.183333 6530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:14.183544 6530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:14.183888 6530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:14.183925 6530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.183955 6530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:14.183979 6530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.189735 6530 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:37153
I20260812 21:11:14.189793 6777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:37153 every 8 connection(s)
I20260812 21:11:14.190127 6530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:14.192438 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 6530
I20260812 21:11:14.192754 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:33495
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:41269
--webserver_port=33457
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:14.199821 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.190837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:38261 (local address 127.11.207.190:41269)
0812 21:11:14.190906 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.190908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.191104 (+ 196us) server_negotiation.cc:407] Connection header received
0812 21:11:14.191594 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.191595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.191604 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.191660 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:14.192805 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.193347 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.194721 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.194939 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.194997 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.197183 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:14.197185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:14.197185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:14.197194 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:14.198498 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.198895 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.198896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.198896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.198953 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.199175 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.199176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.199177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.199294 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:14.199357 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.199626 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.199724 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:11:14.200066 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.190804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:38261)
0812 21:11:14.190978 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.191021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.191515 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.191715 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.191718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.192414 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:14.192708 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.192716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.193467 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.193468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.194624 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.194628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.195013 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.195014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.195149 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.195633 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:14.195642 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:14.197086 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:14.198579 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.198588 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.198590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.198824 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.199006 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.199007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.199007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.199120 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.199356 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:14.199365 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:14.199507 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.199598 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.199768 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260812 21:11:14.200515 6778 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41269
I20260812 21:11:14.200613 6778 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:14.200796 6778 heartbeater.cc:511] Master 127.11.207.190:41269 requested a full tablet report, sending...
I20260812 21:11:14.201189 6172 ts_manager.cc:194] Registered new tserver with Master: 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153)
I20260812 21:11:14.201562 6172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:38261
W20260812 21:11:14.275703 6784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:14.275908 6784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:14.275940 6784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:14.277104 6784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:14.277145 6784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:14.277181 6784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:14.278702 6784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:33495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=33457
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:14.278952 6784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:14.279172 6784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:11:14.284169 6784 server_base.cc:1034] running on GCE node
W20260812 21:11:14.284149 6795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:14.284142 6794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:14.284149 6797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:14.284531 6784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:14.284786 6784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:14.285924 6784 hybrid_clock.cc:648] HybridClock initialized: now 1786569074285909 us; error 29 us; skew 500 ppm
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:14.288322 6784 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:14.288780 6784 webserver.cc:460] Webserver started at http://127.11.207.131:33457/ using document root <none> and password file <none>
I20260812 21:11:14.289005 6784 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:14.289069 6784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:14.290347 6784 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:14.290848 6807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.291015 6784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:14.291103 6784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "763bc8b9ba6c47879c037aca28c69864"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "4a820ce7c2d29ff1b670985bed1fed97"
server_key_iv: "f056964b257c13c568919fd15d1c70b3"
server_key_version: "encryptionkey@0"
I20260812 21:11:14.291421 6784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:14.301465 6784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:14.303161 6784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:14.303295 6784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:14.303503 6784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:14.303793 6784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:14.303826 6784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.303864 6784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:14.303889 6784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.309634 6784 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:33495
I20260812 21:11:14.309670 7035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:33495 every 8 connection(s)
I20260812 21:11:14.310048 6784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:14.318763 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 6784
I20260812 21:11:14.318872 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.310753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:51969 (local address 127.11.207.190:41269)
0812 21:11:14.310818 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.310820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.311037 (+ 217us) server_negotiation.cc:407] Connection header received
0812 21:11:14.311600 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.311601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.311615 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.311669 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:14.312684 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.313185 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.314184 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.314390 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.314446 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.316491 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:14.316493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:14.316493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:14.316500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:14.317661 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.318026 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.318027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.318027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.318076 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.318304 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.318305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.318305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.318403 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:14.318447 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.318723 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.318783 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:11:14.319108 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.310734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:51969)
0812 21:11:14.310916 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.310955 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.311503 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.311739 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.311743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.312327 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:14.312595 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.312600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.313271 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.313272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.314100 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.314103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.314461 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.314462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.314593 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.315022 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:14.315031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:14.316413 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:14.317729 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.317735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.317737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.317962 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.318130 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:14.318131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:14.318131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:14.318257 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:14.318446 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:14.318450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:14.318598 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.318714 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.318887 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260812 21:11:14.319612 7037 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41269
I20260812 21:11:14.319725 7037 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:14.319921 7037 heartbeater.cc:511] Master 127.11.207.190:41269 requested a full tablet report, sending...
I20260812 21:11:14.320334 6172 ts_manager.cc:194] Registered new tserver with Master: 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
I20260812 21:11:14.320840 6172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:51969
I20260812 21:11:14.324939 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.319116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.11.207.190:41269)
0812 21:11:14.319148 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.319150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.319344 (+ 194us) server_negotiation.cc:407] Connection header received
0812 21:11:14.319417 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.319418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.319427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.319486 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:14.320166 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.320905 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.321630 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.321805 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.321861 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.322973 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:14.322974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:14.322974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:14.322979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:14.324071 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.324430 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.324431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.324431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.324477 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:14.324604 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:14.324604 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:14.324605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:14.324689 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:14.324795 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.324823 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.324880 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 21:11:14.325662 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:11:14.328222 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.325970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.11.207.190:41269)
0812 21:11:14.326022 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.326024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.326116 (+ 92us) server_negotiation.cc:407] Connection header received
0812 21:11:14.326190 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.326191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.326228 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.326285 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:14.326394 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.326828 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.327531 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.327750 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.327806 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.327953 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:14.328050 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:11:14.328118 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.328144 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.328161 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:11:14.328497 6172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:11:14.330052 6172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:11:14.332154 6401 tablet_service.cc:1467] Processing CreateTablet for tablet 5d10c67ba2374c3089d52280aa886c5e (DEFAULT_TABLE table=test-table [id=fc335a9c94504335ae91855497a3bbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:14.332402 6401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d10c67ba2374c3089d52280aa886c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:14.333902 6319 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap starting.
I20260812 21:11:14.334223 6319 tablet_bootstrap.cc:654] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:14.334947 6319 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: No bootstrap required, opened a new log
I20260812 21:11:14.334996 6319 ts_tablet_manager.cc:1397] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:14.335117 6319 raft_consensus.cc:348] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.335172 6319 raft_consensus.cc:374] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:14.335206 6319 raft_consensus.cc:729] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a4b9e6e19b489b93102bd04315ab23, State: Initialized, Role: FOLLOWER
I20260812 21:11:14.335242 6319 consensus_queue.cc:260] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.335336 6319 ts_tablet_manager.cc:1428] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.335878 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.331680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:37153 (local address 127.0.0.1:40114)
0812 21:11:14.331884 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.331893 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.331928 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.332193 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.332194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.332196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:14.332235 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.332236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.333579 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.333580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.334701 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.334704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.335621 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.335622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.335685 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.335703 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.335752 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.335797 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260812 21:11:14.336191 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.331804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.11.207.130:37153)
0812 21:11:14.331999 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.332004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.332015 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:11:14.332057 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.332059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.332075 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.332155 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:14.332305 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.333475 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.334788 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.335532 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.335600 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.335651 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.336055 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.336105 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260812 21:11:14.336899 7050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.331771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33495 (local address 127.0.0.1:47586)
0812 21:11:14.331895 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.331906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.331959 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.332845 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.332846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.332849 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:14.333013 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.333015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.334123 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.334124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.335657 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.335660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.336663 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.336664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.336727 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.336731 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.336792 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.336830 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260812 21:11:14.337116 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.332467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47586 (local address 127.11.207.131:33495)
0812 21:11:14.332630 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.332635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.332645 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:11:14.332689 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.332692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.332706 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.332941 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:14.333081 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.333937 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.335747 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.336502 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.336572 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.336627 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.336838 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.336879 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260812 21:11:14.337009 6652 tablet_service.cc:1467] Processing CreateTablet for tablet 5d10c67ba2374c3089d52280aa886c5e (DEFAULT_TABLE table=test-table [id=fc335a9c94504335ae91855497a3bbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:14.337381 6652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d10c67ba2374c3089d52280aa886c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:14.337836 6902 tablet_service.cc:1467] Processing CreateTablet for tablet 5d10c67ba2374c3089d52280aa886c5e (DEFAULT_TABLE table=test-table [id=fc335a9c94504335ae91855497a3bbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:14.338126 6902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d10c67ba2374c3089d52280aa886c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:14.340297 7058 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Bootstrap starting.
I20260812 21:11:14.341046 7058 tablet_bootstrap.cc:654] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:14.341037 7059 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Bootstrap starting.
I20260812 21:11:14.341536 7058 log.cc:824] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:14.341558 7059 tablet_bootstrap.cc:654] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:14.342113 7059 log.cc:824] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:14.342274 7058 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: No bootstrap required, opened a new log
I20260812 21:11:14.342352 7058 ts_tablet_manager.cc:1397] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:14.342804 7059 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: No bootstrap required, opened a new log
I20260812 21:11:14.342880 7059 ts_tablet_manager.cc:1397] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:11:14.343918 7058 raft_consensus.cc:348] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.344049 7058 raft_consensus.cc:374] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:14.344069 7058 raft_consensus.cc:729] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bccc4bfaa684ea2a28ce833f909fd7e, State: Initialized, Role: FOLLOWER
I20260812 21:11:14.344172 7058 consensus_queue.cc:260] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.344466 6778 heartbeater.cc:503] Master 127.11.207.190:41269 was elected leader, sending a full tablet report...
I20260812 21:11:14.344494 7058 ts_tablet_manager.cc:1428] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:14.344581 7059 raft_consensus.cc:348] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.344722 7059 raft_consensus.cc:374] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:14.344760 7059 raft_consensus.cc:729] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763bc8b9ba6c47879c037aca28c69864, State: Initialized, Role: FOLLOWER
I20260812 21:11:14.344877 7059 consensus_queue.cc:260] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.345132 7059 ts_tablet_manager.cc:1428] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:14.345158 7037 heartbeater.cc:503] Master 127.11.207.190:41269 was elected leader, sending a full tablet report...
W20260812 21:11:14.440694 6779 tablet.cc:2369] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:14.559842 6523 tablet.cc:2369] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:14.560575 7044 tablet.cc:2369] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:14.749466 6529 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:14.749630 6529 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.750054 6529 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
I20260812 21:11:14.753011 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.750346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:57233 (local address 127.11.207.130:37153)
0812 21:11:14.750409 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.750412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.750492 (+ 80us) server_negotiation.cc:407] Connection header received
0812 21:11:14.750603 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.750605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.750614 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.750655 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:14.750805 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.751306 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.752272 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.752665 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.752733 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.752743 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.752901 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.752940 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:11:14.753024 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.750253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:37153 (local address 127.11.207.129:57233)
0812 21:11:14.750406 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.750419 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.750517 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.750671 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.750671 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.750675 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:14.750742 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.750744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.751387 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.751388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.752177 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.752179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.752750 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.752751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.752835 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.752838 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.752885 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.752930 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260812 21:11:14.753010 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.750549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:42993 (local address 127.11.207.131:33495)
0812 21:11:14.750592 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.750594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.750622 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:11:14.750699 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.750700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.750709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.750751 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:14.750897 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.751353 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.752275 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.752670 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.752735 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.752744 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.752897 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.752936 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:11:14.753037 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.750447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33495 (local address 127.11.207.129:42993)
0812 21:11:14.750563 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.750572 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:14.750631 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:14.750764 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:14.750765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:14.750767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:14.750831 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.750833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.751427 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.751428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.752186 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:14.752188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.752748 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.752749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:14.752809 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.752833 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 21:11:14.752882 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:11:14.752930 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":6,"thread_start_us":44,"threads_started":1}
I20260812 21:11:14.753391 6948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d10c67ba2374c3089d52280aa886c5e" candidate_uuid: "72a4b9e6e19b489b93102bd04315ab23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "763bc8b9ba6c47879c037aca28c69864" is_pre_election: true
I20260812 21:11:14.753579 6948 raft_consensus.cc:2454] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72a4b9e6e19b489b93102bd04315ab23 in term 0.
I20260812 21:11:14.753556 6692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d10c67ba2374c3089d52280aa886c5e" candidate_uuid: "72a4b9e6e19b489b93102bd04315ab23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" is_pre_election: true
I20260812 21:11:14.753690 6692 raft_consensus.cc:2454] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72a4b9e6e19b489b93102bd04315ab23 in term 0.
I20260812 21:11:14.753846 6315 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23, 763bc8b9ba6c47879c037aca28c69864; no voters:
I20260812 21:11:14.753978 6529 raft_consensus.cc:2781] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:11:14.754036 6529 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:11:14.754060 6529 raft_consensus.cc:3037] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:14.754756 6529 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.754889 6529 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 1 election: Requested vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
I20260812 21:11:14.755084 6692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d10c67ba2374c3089d52280aa886c5e" candidate_uuid: "72a4b9e6e19b489b93102bd04315ab23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
I20260812 21:11:14.755118 6948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d10c67ba2374c3089d52280aa886c5e" candidate_uuid: "72a4b9e6e19b489b93102bd04315ab23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "763bc8b9ba6c47879c037aca28c69864"
I20260812 21:11:14.755165 6692 raft_consensus.cc:3037] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:14.755194 6948 raft_consensus.cc:3037] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:14.755916 6692 raft_consensus.cc:2454] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72a4b9e6e19b489b93102bd04315ab23 in term 1.
I20260812 21:11:14.755920 6948 raft_consensus.cc:2454] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72a4b9e6e19b489b93102bd04315ab23 in term 1.
I20260812 21:11:14.756078 6315 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bccc4bfaa684ea2a28ce833f909fd7e, 72a4b9e6e19b489b93102bd04315ab23; no voters:
I20260812 21:11:14.756165 6529 raft_consensus.cc:2781] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:14.756274 6529 raft_consensus.cc:686] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 LEADER]: Becoming Leader. State: Replica: 72a4b9e6e19b489b93102bd04315ab23, State: Running, Role: LEADER
I20260812 21:11:14.756336 6529 consensus_queue.cc:237] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:14.757081 6172 catalog_manager.cc:5495] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129). New cstate: current_term: 1 leader_uuid: "72a4b9e6e19b489b93102bd04315ab23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } health_report { overall_health: HEALTHY } } }
I20260812 21:11:14.775449 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.773226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.11.207.129:33795)
0812 21:11:14.773391 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:14.773393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:14.773401 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:11:14.773442 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:14.773443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:14.773463 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:14.773541 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:14.773635 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.774102 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.774798 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:14.774994 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:14.775038 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:14.775160 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:14.775270 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:11:14.775327 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:14.775362 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 21:11:14.775379 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260812 21:11:14.777071 6948 raft_consensus.cc:1261] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Refusing update from remote peer 72a4b9e6e19b489b93102bd04315ab23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:14.777084 6692 raft_consensus.cc:1261] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Refusing update from remote peer 72a4b9e6e19b489b93102bd04315ab23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:14.777407 6529 consensus_queue.cc:1035] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:14.777566 7069 consensus_queue.cc:1035] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:14.780045 7077 mvcc.cc:205] Tried to move back new op lower bound from 7317786930284109824 to 7317786930202636288. Current Snapshot: MvccSnapshot[applied={T|T < 7317786930284109824}]
I20260812 21:11:14.780616 7078 mvcc.cc:205] Tried to move back new op lower bound from 7317786930284109824 to 7317786930202636288. Current Snapshot: MvccSnapshot[applied={T|T < 7317786930284109824}]
I20260812 21:11:14.783069 6172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:14.783183 6172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:14.783951 6172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Sending DeleteTablet for 3 replicas of tablet 5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:14.784245 6401 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d10c67ba2374c3089d52280aa886c5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:14 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:56870
I20260812 21:11:14.784374 6652 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d10c67ba2374c3089d52280aa886c5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:14 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:40114
I20260812 21:11:14.784482 6902 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d10c67ba2374c3089d52280aa886c5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:14 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:47586
I20260812 21:11:14.784515 7090 tablet_replica.cc:330] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: stopping tablet replica
I20260812 21:11:14.784571 7090 raft_consensus.cc:2229] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:11:14.784638 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 6238
I20260812 21:11:14.784668 7090 raft_consensus.cc:2258] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:14.796414 7094 tablet_replica.cc:330] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: stopping tablet replica
I20260812 21:11:14.796589 7094 raft_consensus.cc:2229] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:14.796733 7094 raft_consensus.cc:2258] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:14.796952 7095 tablet_replica.cc:330] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: stopping tablet replica
I20260812 21:11:14.797075 7095 raft_consensus.cc:2229] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:14.797187 7095 raft_consensus.cc:2258] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:14.797398 7094 ts_tablet_manager.cc:1905] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:14.797626 7095 ts_tablet_manager.cc:1905] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:14.799144 7095 ts_tablet_manager.cc:1918] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:14.799145 7094 ts_tablet_manager.cc:1918] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:14.799221 7094 log.cc:1196] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:14.799221 7095 log.cc:1196] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:14.799494 7095 ts_tablet_manager.cc:1939] T 5d10c67ba2374c3089d52280aa886c5e P 763bc8b9ba6c47879c037aca28c69864: Deleting consensus metadata
I20260812 21:11:14.799523 7094 ts_tablet_manager.cc:1939] T 5d10c67ba2374c3089d52280aa886c5e P 6bccc4bfaa684ea2a28ce833f909fd7e: Deleting consensus metadata
I20260812 21:11:14.799988 6151 catalog_manager.cc:4841] TS 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): tablet 5d10c67ba2374c3089d52280aa886c5e (table test-table [id=fc335a9c94504335ae91855497a3bbbd]) successfully deleted
I20260812 21:11:14.800127 6151 catalog_manager.cc:4841] TS 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): tablet 5d10c67ba2374c3089d52280aa886c5e (table test-table [id=fc335a9c94504335ae91855497a3bbbd]) successfully deleted
W20260812 21:11:14.802065 6147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:33795 (error 108)
I20260812 21:11:14.802369 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 6530
I20260812 21:11:14.810348 7050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:14.802420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:33795 (local address 127.0.0.1:56880)
0812 21:11:14.810232 (+ 7812us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:14.810305 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:33795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7767}
W20260812 21:11:14.810529 6147 catalog_manager.cc:4584] TS 72a4b9e6e19b489b93102bd04315ab23 (127.11.207.129:33795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d10c67ba2374c3089d52280aa886c5e: Network error: Client connection negotiation failed: client connection to 127.11.207.129:33795: connect: Connection refused (error 111)
I20260812 21:11:14.815659 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 6784
I20260812 21:11:14.828670 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 6110
I20260812 21:11:14.841120 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41269
--webserver_interface=127.11.207.190
--webserver_port=45469
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:14.925819 7097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:14.926038 7097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:14.926061 7097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:14.927187 7097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:14.927224 7097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:14.927237 7097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:14.927250 7097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:14.927260 7097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:14.928771 7097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=45469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:14.929003 7097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:14.929205 7097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:14.934091 7106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:14.934167 7119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:14.934108 7097 server_base.cc:1034] running on GCE node
W20260812 21:11:14.934094 7105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:14.934520 7097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:14.934842 7097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:14.935982 7097 hybrid_clock.cc:648] HybridClock initialized: now 1786569074935967 us; error 36 us; skew 500 ppm
Aug 12 21:11:14 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569074, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:14.938405 7097 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:14.938840 7097 webserver.cc:460] Webserver started at http://127.11.207.190:45469/ using document root <none> and password file <none>
I20260812 21:11:14.939040 7097 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:14.939085 7097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:14.940378 7097 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:11:14.941004 7141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:14.941162 7097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:14.941246 7097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "f9fc3a8af1048618e6ac98c00308b97b"
server_key_iv: "8fb6308ee60d351342de7ac6b2831bd5"
server_key_version: "encryptionkey@0"
I20260812 21:11:14.941601 7097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:14.972853 7097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:14.974771 7097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:14.974913 7097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:14.979058 7097 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41269
I20260812 21:11:14.979099 7243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41269 every 8 connection(s)
I20260812 21:11:14.979465 7097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:14.981170 7248 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:11:14.981873 7248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap starting.
I20260812 21:11:14.987382 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 7097
I20260812 21:11:14.987665 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:33795
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:41269
--webserver_port=46647
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:14.991220 7248 log.cc:824] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:14.993652 7248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:14.993942 7248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a: Bootstrap complete.
I20260812 21:11:14.996129 7248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:14.996402 7248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f70339eecbb4ee9b29d8d1bf46b7e4a, State: Initialized, Role: FOLLOWER
I20260812 21:11:14.996568 7248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:14.996634 7248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:14.996718 7248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:14.996802 7248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:11:14.997851 7248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:14.997951 7248 leader_election.cc:302] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f70339eecbb4ee9b29d8d1bf46b7e4a; no voters:
I20260812 21:11:14.998116 7248 leader_election.cc:288] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 21:11:14.998170 7252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 3 FOLLOWER]: Leader election won for term 3
I20260812 21:11:14.998286 7252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [term 3 LEADER]: Becoming Leader. State: Replica: 1f70339eecbb4ee9b29d8d1bf46b7e4a, State: Running, Role: LEADER
I20260812 21:11:14.998414 7248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:14.998395 7252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } }
I20260812 21:11:14.998761 7256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f70339eecbb4ee9b29d8d1bf46b7e4a. Latest consensus state: current_term: 3 leader_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } } }
I20260812 21:11:14.998916 7256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:14.999104 7255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f70339eecbb4ee9b29d8d1bf46b7e4a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41269 } } }
I20260812 21:11:14.999203 7255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f70339eecbb4ee9b29d8d1bf46b7e4a [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:14.999243 7273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:15.000108 7273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:15.000219 7273 catalog_manager.cc:1259] Loaded cluster ID: 0d582dfb024348ac8d5bc7fde1c17d52
I20260812 21:11:15.000257 7273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 21:11:15.000631 7273 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0d404b09ad google::LogMessage::Fail() at ??:0
@ 0x7f0d404b388a google::LogMessage::SendToLog() at ??:0
@ 0x7f0d404b04de google::LogMessage::Flush() at ??:0
@ 0x7f0d404b1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0d483d10f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0d483da8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0d4237e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0d42373954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0d431d76db start_thread at ??:0
@ 0x7f0d3efae61f clone at ??:0
W20260812 21:11:15.083894 7250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:15.084103 7250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:15.084126 7250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:15.085323 7250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:15.085366 7250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:15.085392 7250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:15.086834 7250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:33795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=46647
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:15.087052 7250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:15.087270 7250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:15.092399 7330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:15.092450 7331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:15.092499 7338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:15.092638 7250 server_base.cc:1034] running on GCE node
I20260812 21:11:15.092926 7250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:15.093256 7250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:15.094455 7250 hybrid_clock.cc:648] HybridClock initialized: now 1786569075094432 us; error 36 us; skew 500 ppm
Aug 12 21:11:15 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569075, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:15.097642 7250 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:15.098292 7250 webserver.cc:460] Webserver started at http://127.11.207.129:46647/ using document root <none> and password file <none>
I20260812 21:11:15.098575 7250 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:15.098658 7250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:15.100508 7250 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:11:15.101325 7349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.101495 7250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:15.101606 7250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "72a4b9e6e19b489b93102bd04315ab23"
format_stamp: "Formatted at 2026-08-12 21:11:12 on dist-test-slave-lkx2"
server_key: "af6e5e7e9a5264d6a828027bcecc259e"
server_key_iv: "7327581df8d3c824b40e5826a96060eb"
server_key_version: "encryptionkey@0"
I20260812 21:11:15.102092 7250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:15.124805 7250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:15.127449 7250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:15.127641 7250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:15.127990 7250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:15.128597 7356 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:11:15.130079 7250 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:11:15.130156 7250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:11:15.130198 7250 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:11:15.131165 7250 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:11:15.131218 7250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:15.131258 7356 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap starting.
I20260812 21:11:15.139881 7250 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:33795
I20260812 21:11:15.139990 7584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:33795 every 8 connection(s)
I20260812 21:11:15.140442 7250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:11:15.142290 7356 log.cc:824] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:15.143774 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 7250
I20260812 21:11:15.144089 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:37153
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:41269
--webserver_port=45671
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:15.144757 7356 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:15.145057 7356 tablet_bootstrap.cc:492] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Bootstrap complete.
I20260812 21:11:15.146335 7356 ts_tablet_manager.cc:1397] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260812 21:11:15.147403 7356 raft_consensus.cc:348] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:15.147639 7356 raft_consensus.cc:729] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a4b9e6e19b489b93102bd04315ab23, State: Initialized, Role: FOLLOWER
I20260812 21:11:15.147789 7356 consensus_queue.cc:260] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:15.148090 7356 ts_tablet_manager.cc:1428] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260812 21:11:15.219534 7590 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:11:15.141186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:35767)
0812 21:11:15.141365 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.141413 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:15.142078 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:15.219313 (+ 77235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
W20260812 21:11:15.219887 7588 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:41269 (0 consecutive failures): Network error: Failed to ping master at 127.11.207.190:41269: Client connection negotiation failed: client connection to 127.11.207.190:41269: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:11:15.220258 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.220123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:56401)
0812 21:11:15.220178 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.220232 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:11:15.220616 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.220523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:39573)
0812 21:11:15.220564 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.220594 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 21:11:15.220779 7588 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:11:15.230602 7595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:15.230798 7595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:15.230821 7595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:15.231971 7595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:15.232013 7595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:15.232038 7595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:15.233656 7595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:37153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=45671
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:15.233904 7595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:15.234143 7595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:15.239118 7615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:15.239109 7618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:15.239190 7595 server_base.cc:1034] running on GCE node
W20260812 21:11:15.239193 7613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:15.239578 7595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:15.239794 7595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:15.240918 7595 hybrid_clock.cc:648] HybridClock initialized: now 1786569075240909 us; error 28 us; skew 500 ppm
Aug 12 21:11:15 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569075, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:15.243268 7595 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:15.243693 7595 webserver.cc:460] Webserver started at http://127.11.207.130:45671/ using document root <none> and password file <none>
I20260812 21:11:15.243921 7595 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:15.243970 7595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:15.245379 7595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:15.245913 7625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.246083 7595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:15.246167 7595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "6bccc4bfaa684ea2a28ce833f909fd7e"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "ce9eddc553d020c5d7fb6ab18231f8cd"
server_key_iv: "8330c695ef20c582ded9751dd3e88879"
server_key_version: "encryptionkey@0"
I20260812 21:11:15.246531 7595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:15.262090 7595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:15.264190 7595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:15.264328 7595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:15.264590 7595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:15.264910 7595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:15.264945 7595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.264974 7595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:15.264997 7595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.270720 7595 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:37153
I20260812 21:11:15.270771 7859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:37153 every 8 connection(s)
I20260812 21:11:15.271104 7595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:15.271586 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 7595
I20260812 21:11:15.271860 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:33495
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:41269
--webserver_port=33457
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:15.272245 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.271785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:39125)
0812 21:11:15.271953 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.272060 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
W20260812 21:11:15.272701 7860 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:41269 (0 consecutive failures): Network error: Failed to ping master at 127.11.207.190:41269: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
I20260812 21:11:15.272989 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.272888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:46031)
0812 21:11:15.272935 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.272967 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:11:15.274156 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.274067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:57567)
0812 21:11:15.274107 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.274131 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:11:15.274281 7860 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:11:15.357191 7863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:15.357409 7863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:15.357434 7863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:15.358593 7863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:15.358640 7863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:15.358665 7863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:15.360175 7863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:33495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=33457
--tserver_master_addrs=127.11.207.190:41269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:15.360442 7863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:15.360663 7863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:15.365653 7872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:15.365746 7879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:15.365689 7870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:15.365877 7863 server_base.cc:1034] running on GCE node
I20260812 21:11:15.366072 7863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:15.366290 7863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:15.367419 7863 hybrid_clock.cc:648] HybridClock initialized: now 1786569075367409 us; error 27 us; skew 500 ppm
Aug 12 21:11:15 dist-test-slave-lkx2 krb5kdc[5067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569075, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:15.370060 7863 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:15.370476 7863 webserver.cc:460] Webserver started at http://127.11.207.131:33457/ using document root <none> and password file <none>
I20260812 21:11:15.370668 7863 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:15.370708 7863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:15.371982 7863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:15.372581 7888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.372751 7863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:15.372851 7863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "763bc8b9ba6c47879c037aca28c69864"
format_stamp: "Formatted at 2026-08-12 21:11:13 on dist-test-slave-lkx2"
server_key: "4a820ce7c2d29ff1b670985bed1fed97"
server_key_iv: "f056964b257c13c568919fd15d1c70b3"
server_key_version: "encryptionkey@0"
I20260812 21:11:15.373198 7863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:15.383487 7863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:15.385157 7863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:15.385263 7863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:15.385432 7863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:15.385675 7863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:15.385712 7863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.385732 7863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:15.385751 7863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:15.391330 7863 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:33495
I20260812 21:11:15.391398 8103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:33495 every 8 connection(s)
I20260812 21:11:15.391683 7863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:11:15.392731 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.392316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:50441)
0812 21:11:15.392499 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.392574 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
W20260812 21:11:15.393181 8106 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:41269 (0 consecutive failures): Network error: Failed to ping master at 127.11.207.190:41269: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
I20260812 21:11:15.393440 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.393351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:49751)
0812 21:11:15.393398 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.393423 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 21:11:15.393637 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:15.393581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:51123)
0812 21:11:15.393609 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:15.393626 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 21:11:15.393714 8106 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:11:15.398247 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 7863
I20260812 21:11:16.221366 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.221115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:49111)
0812 21:11:16.221245 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:16.221285 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260812 21:11:16.274830 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.274583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:39083)
0812 21:11:16.274722 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:16.274764 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260812 21:11:16.394310 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.394053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:40383)
0812 21:11:16.394194 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:16.394240 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260812 21:11:16.829962 8118 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:16.830119 8118 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:16.830507 8118 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
Aug 12 21:11:16 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569075, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.131@KRBTEST.COM
Aug 12 21:11:16 dist-test-slave-lkx2 krb5kdc[5067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569075, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.130@KRBTEST.COM
I20260812 21:11:16.842828 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.830779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:46503 (local address 127.11.207.130:37153)
0812 21:11:16.830967 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:16.830980 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:16.831001 (+ 21us) server_negotiation.cc:407] Connection header received
0812 21:11:16.831585 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:16.831588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:16.831650 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:16.831867 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:16.832912 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.834373 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.835909 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.836528 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.836647 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:16.839696 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:16.839701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:16.839701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:16.839712 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:16.841368 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841713 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:16.841717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:16.841717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:16.841807 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841998 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:16.841999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:16.841999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:16.842245 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:16.842315 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:16.842581 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 21:11:16.842715 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":56,"threads_started":1}
I20260812 21:11:16.842844 8128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.830988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:38919 (local address 127.11.207.131:33495)
0812 21:11:16.831111 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:16.831118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:16.831149 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:11:16.831696 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:16.831700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:16.831755 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:16.831871 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:16.832912 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.834373 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.835872 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.836528 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.836643 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:16.839361 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:16.839367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:16.839367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:16.839381 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:16.841162 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841645 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:16.841649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:16.841649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:16.841751 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841999 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:16.842000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:16.842001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:16.842245 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:16.842317 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:16.842587 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 21:11:16.842708 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260812 21:11:16.842823 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.830859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:33495 (local address 127.11.207.129:38919)
0812 21:11:16.831112 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:16.831120 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:16.831159 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:16.831932 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:16.831935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:16.832467 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:16.832746 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:16.832754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.834499 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.834500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:16.835784 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:16.835787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.836618 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.836618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:16.836753 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:16.837380 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:16.837391 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:16.839243 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:16.841269 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:16.841279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:16.841548 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:16.841835 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:16.841836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:16.841941 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:16.842313 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:16.842314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:16.842433 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 21:11:16.842521 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:11:16.842721 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":82,"thread_start_us":41,"threads_started":1}
I20260812 21:11:16.842916 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:16.830660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:37153 (local address 127.11.207.129:46503)
0812 21:11:16.830779 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:16.830789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:16.830871 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:16.831932 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:16.831935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:16.832467 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:16.832746 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:16.832754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.834512 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.834513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:16.835784 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:16.835787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:16.836617 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:16.836618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:16.836753 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:16.837395 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:16.837399 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:16.839585 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:16.841438 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:16.841440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:16.841634 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:16.841866 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:16.841867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:16.841867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:16.841943 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:16.842306 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:16.842313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:16.842433 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 21:11:16.842521 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:11:16.842794 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":157,"thread_start_us":83,"threads_started":1}
W20260812 21:11:16.843696 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
W20260812 21:11:16.843788 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:16.843823 7353 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23; no voters: 6bccc4bfaa684ea2a28ce833f909fd7e, 763bc8b9ba6c47879c037aca28c69864
I20260812 21:11:16.843951 8118 raft_consensus.cc:2726] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:11:17.222011 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:17.221876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:59705)
0812 21:11:17.221926 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:17.221968 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:11:17.275542 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:17.275302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:48805)
0812 21:11:17.275440 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:17.275491 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260812 21:11:17.395016 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:17.394793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:53563)
0812 21:11:17.394930 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:17.394974 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260812 21:11:18.222848 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:18.222481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:50385)
0812 21:11:18.222745 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:18.222797 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":78,"threads_started":1}
I20260812 21:11:18.276221 8134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:18.275964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:41963)
0812 21:11:18.276112 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:18.276155 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260812 21:11:18.395766 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:18.395485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:44753)
0812 21:11:18.395662 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:18.395714 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":87,"threads_started":1}
I20260812 21:11:18.534628 8140 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:18.534730 8140 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:18.534883 8140 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
W20260812 21:11:18.535483 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
W20260812 21:11:18.535567 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:18.535585 7353 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23; no voters: 6bccc4bfaa684ea2a28ce833f909fd7e, 763bc8b9ba6c47879c037aca28c69864
I20260812 21:11:18.535719 8140 raft_consensus.cc:2726] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:11:19.223580 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:19.223361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:44857)
0812 21:11:19.223486 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:19.223529 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260812 21:11:19.276943 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:19.276719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:48235)
0812 21:11:19.276855 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:19.276898 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260812 21:11:19.396528 8146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:19.396308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:39593)
0812 21:11:19.396438 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:19.396482 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260812 21:11:20.224376 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:20.224094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:60981)
0812 21:11:20.224256 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:20.224299 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260812 21:11:20.277582 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:20.277371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:44309)
0812 21:11:20.277498 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:20.277537 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260812 21:11:20.397241 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:20.397006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:41969)
0812 21:11:20.397149 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:20.397193 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260812 21:11:21.225054 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:21.224777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:60893)
0812 21:11:21.224945 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:21.224997 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
I20260812 21:11:21.278338 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:21.278064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:60369)
0812 21:11:21.278213 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:21.278265 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260812 21:11:21.397948 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:21.397713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:49315)
0812 21:11:21.397859 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:21.397903 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260812 21:11:21.576231 8156 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:21.576335 8156 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:21.576517 8156 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
W20260812 21:11:21.576943 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
W20260812 21:11:21.577009 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:21.577023 7353 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23; no voters: 6bccc4bfaa684ea2a28ce833f909fd7e, 763bc8b9ba6c47879c037aca28c69864
I20260812 21:11:21.577086 8156 raft_consensus.cc:2726] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:11:22.225693 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:22.225483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:42419)
0812 21:11:22.225612 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:22.225643 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260812 21:11:22.278985 8160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:22.278759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:60873)
0812 21:11:22.278891 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:22.278935 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260812 21:11:22.398648 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:22.398401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:55141)
0812 21:11:22.398530 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:22.398578 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260812 21:11:23.226363 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:23.226125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:47113)
0812 21:11:23.226274 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:23.226313 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260812 21:11:23.279553 8166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:23.279364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:43955)
0812 21:11:23.279475 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:23.279511 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260812 21:11:23.399380 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:23.399142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:56963)
0812 21:11:23.399292 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:23.399333 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260812 21:11:24.227139 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:24.226922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:53563)
0812 21:11:24.227053 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:24.227088 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260812 21:11:24.280205 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:24.279993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:50169)
0812 21:11:24.280123 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:24.280156 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260812 21:11:24.400167 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:24.399891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:57331)
0812 21:11:24.400069 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:24.400115 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
I20260812 21:11:25.073705 8174 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:25.073814 8174 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:25.073956 8174 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
W20260812 21:11:25.074350 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
W20260812 21:11:25.074450 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:25.074487 7353 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23; no voters: 6bccc4bfaa684ea2a28ce833f909fd7e, 763bc8b9ba6c47879c037aca28c69864
I20260812 21:11:25.074576 8174 raft_consensus.cc:2726] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:11:25.227993 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:25.227711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:53461)
0812 21:11:25.227871 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:25.227921 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260812 21:11:25.280992 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:25.280731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:57131)
0812 21:11:25.280896 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:25.280944 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260812 21:11:25.401048 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:25.400650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:55823)
0812 21:11:25.400790 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:25.400990 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260812 21:11:26.228925 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:26.228632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:43273)
0812 21:11:26.228818 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:26.228872 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260812 21:11:26.281783 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:26.281493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:48987)
0812 21:11:26.281671 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:26.281727 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260812 21:11:26.401770 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:26.401522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:50045)
0812 21:11:26.401680 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:26.401718 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260812 21:11:26.857744 8189 raft_consensus.cc:482] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:26.857843 8189 raft_consensus.cc:504] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bccc4bfaa684ea2a28ce833f909fd7e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 37153 } } peers { permanent_uuid: "763bc8b9ba6c47879c037aca28c69864" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 33495 } } peers { permanent_uuid: "72a4b9e6e19b489b93102bd04315ab23" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 33795 } }
I20260812 21:11:26.857985 8189 leader_election.cc:288] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153), 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495)
W20260812 21:11:26.858424 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6bccc4bfaa684ea2a28ce833f909fd7e (127.11.207.130:37153): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
W20260812 21:11:26.858500 7353 leader_election.cc:341] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 763bc8b9ba6c47879c037aca28c69864 (127.11.207.131:33495): Not found: Tablet not found: 5d10c67ba2374c3089d52280aa886c5e
I20260812 21:11:26.858520 7353 leader_election.cc:302] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72a4b9e6e19b489b93102bd04315ab23; no voters: 6bccc4bfaa684ea2a28ce833f909fd7e, 763bc8b9ba6c47879c037aca28c69864
I20260812 21:11:26.858613 8189 raft_consensus.cc:2726] T 5d10c67ba2374c3089d52280aa886c5e P 72a4b9e6e19b489b93102bd04315ab23 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:11:27.229679 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:27.229429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:53105)
0812 21:11:27.229584 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:27.229629 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260812 21:11:27.282498 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:27.282289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:47361)
0812 21:11:27.282406 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:27.282450 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260812 21:11:27.402535 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:27.402248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:32847)
0812 21:11:27.402410 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:27.402477 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260812 21:11:28.230688 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:28.230399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:39369)
0812 21:11:28.230592 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:28.230639 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":97,"threads_started":1}
I20260812 21:11:28.283216 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:28.282978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:40025)
0812 21:11:28.283122 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:28.283167 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260812 21:11:28.403314 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:28.403065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:52553)
0812 21:11:28.403201 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:28.403248 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260812 21:11:29.231467 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:29.231240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:54589)
0812 21:11:29.231372 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:29.231412 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260812 21:11:29.283987 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:29.283731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:53527)
0812 21:11:29.283897 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:29.283938 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260812 21:11:29.404042 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:29.403780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:58413)
0812 21:11:29.403941 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:29.403990 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260812 21:11:30.232327 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:30.232058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.129:47149)
0812 21:11:30.232211 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:30.232277 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260812 21:11:30.284845 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:30.284593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.130:37417)
0812 21:11:30.284745 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:30.284794 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260812 21:11:30.402482 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 7250
I20260812 21:11:30.408567 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:30.408224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41269 (local address 127.11.207.131:50077)
0812 21:11:30.408475 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:30.408515 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:41269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":68,"threads_started":1}
I20260812 21:11:30.415948 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 7595
I20260812 21:11:30.430240 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 7863
2026-08-12T21:11:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19134 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:11:30 dist-test-slave-lkx2 krb5kdc[8222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:11:30 dist-test-slave-lkx2 krb5kdc[8222](info): set up 2 sockets
Aug 12 21:11:30 dist-test-slave-lkx2 krb5kdc[8222](info): commencing operation
krb5kdc: starting...
W20260812 21:11:32.062988 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.599s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:11:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:11:32.185259 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44453
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:44453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:32.267581 8248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:32.267798 8248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:32.267828 8248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:32.269030 8248 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 21:11:32.269073 8248 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 21:11:32.269090 8248 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 21:11:32.269105 8248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:32.269121 8248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:32.269137 8248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:32.269162 8248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:32.269194 8248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:32.270695 8248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:44453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:32.270948 8248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:32.271153 8248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:32.276185 8265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.276240 8259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.276240 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:32.276417 8248 server_base.cc:1034] running on GCE node
I20260812 21:11:32.276736 8248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:32.277015 8248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:32.278185 8248 hybrid_clock.cc:648] HybridClock initialized: now 1786569092278170 us; error 32 us; skew 500 ppm
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:32.280732 8248 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:32.281121 8248 webserver.cc:460] Webserver started at http://127.11.207.190:33633/ using document root <none> and password file <none>
I20260812 21:11:32.281312 8248 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:32.281351 8248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:32.281442 8248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:32.282603 8248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "ae5366270f46483398ab62d4e51646df"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "26d5b65e5560bb2f5934746b0767857c"
server_key_iv: "d1a18ea639cc8e9603449338ee8ea4b5"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.282989 8248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "ae5366270f46483398ab62d4e51646df"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "26d5b65e5560bb2f5934746b0767857c"
server_key_iv: "d1a18ea639cc8e9603449338ee8ea4b5"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.284361 8248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:11:32.285081 8280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.285259 8248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:32.285351 8248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "ae5366270f46483398ab62d4e51646df"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "26d5b65e5560bb2f5934746b0767857c"
server_key_iv: "d1a18ea639cc8e9603449338ee8ea4b5"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.285420 8248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:32.298038 8248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:32.299852 8248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:32.299993 8248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:32.303966 8248 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:44453
I20260812 21:11:32.304015 8378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:44453 every 8 connection(s)
I20260812 21:11:32.304366 8248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:32.305233 8379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:32.307538 8379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap starting.
I20260812 21:11:32.308137 8379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:32.308468 8379 log.cc:824] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:32.309224 8379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: No bootstrap required, opened a new log
I20260812 21:11:32.310956 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 8248
I20260812 21:11:32.310889 8379 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:32.311019 8379 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:32.311041 8379 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5366270f46483398ab62d4e51646df, State: Initialized, Role: FOLLOWER
I20260812 21:11:32.311054 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:11:32.311173 8379 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:32.311244 8379 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:32.311280 8379 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:32.311271 12094 external_mini_cluster.cc:1428] Setting key 0cff9c747f4a9105731e5e412d4daf56
I20260812 21:11:32.311328 8379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:32.312008 8379 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:32.312112 8379 leader_election.cc:302] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae5366270f46483398ab62d4e51646df; no voters:
I20260812 21:11:32.312294 8379 leader_election.cc:288] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:11:32.312335 8388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:32.312474 8388 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 LEADER]: Becoming Leader. State: Replica: ae5366270f46483398ab62d4e51646df, State: Running, Role: LEADER
I20260812 21:11:32.312628 8379 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:32.312618 8388 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:32.313254 8390 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae5366270f46483398ab62d4e51646df. Latest consensus state: current_term: 1 leader_uuid: "ae5366270f46483398ab62d4e51646df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } } }
I20260812 21:11:32.313337 8390 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:32.313597 8392 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae5366270f46483398ab62d4e51646df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } } }
I20260812 21:11:32.313725 8392 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:32.313911 8406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:32.314459 8406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:32.316576 8406 catalog_manager.cc:1347] Generated new cluster ID: 449d34439dd94aadaf7c267c00e48144
I20260812 21:11:32.316627 8406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:32.322818 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.311799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.11.207.190:44453)
0812 21:11:32.312492 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.312503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.312536 (+ 33us) server_negotiation.cc:407] Connection header received
0812 21:11:32.312669 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.312677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.312737 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.312884 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:32.313727 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.314787 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.315791 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.316291 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.316421 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.319078 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:32.319082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:32.319083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:32.319095 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:32.321002 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.321375 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.321378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.321378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.321476 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.321641 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.321642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.321642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.321921 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:32.322046 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.322455 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.322589 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":516,"threads_started":1}
I20260812 21:11:32.339252 8406 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:11:32.339814 8406 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:32.348142 8406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Generated new TSK 0
I20260812 21:11:32.348366 8406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:11:32.381340 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:44453
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:32.485122 8442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:32.485328 8442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:32.485350 8442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:32.486470 8442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:32.486513 8442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:32.486539 8442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:32.488030 8442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:32.488245 8442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:32.488503 8442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:11:32.493587 8442 server_base.cc:1034] running on GCE node
W20260812 21:11:32.493522 8457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.493530 8454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.493521 8460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:32.493969 8442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:32.494225 8442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:32.495375 8442 hybrid_clock.cc:648] HybridClock initialized: now 1786569092495360 us; error 28 us; skew 500 ppm
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:32.497902 8442 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:32.498299 8442 webserver.cc:460] Webserver started at http://127.11.207.129:45607/ using document root <none> and password file <none>
I20260812 21:11:32.498473 8442 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:32.498512 8442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:32.498600 8442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:32.499584 8442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "665d9eb8786f406f94a5b9245fc02e0d"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "f7fa5318979bb05c7e7d9a64f99ea7ff"
server_key_iv: "6bf370eb9cfb765e34c2618fdccbcd06"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.499881 8442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "665d9eb8786f406f94a5b9245fc02e0d"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "f7fa5318979bb05c7e7d9a64f99ea7ff"
server_key_iv: "6bf370eb9cfb765e34c2618fdccbcd06"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.501150 8442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:11:32.501873 8473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.502048 8442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:32.502136 8442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "665d9eb8786f406f94a5b9245fc02e0d"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "f7fa5318979bb05c7e7d9a64f99ea7ff"
server_key_iv: "6bf370eb9cfb765e34c2618fdccbcd06"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.502197 8442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:32.510556 8442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:32.512641 8442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:32.512771 8442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:32.513003 8442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:32.513345 8442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:32.513396 8442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.513434 8442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:32.513458 8442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.519260 8442 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:37865
I20260812 21:11:32.519327 8693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:37865 every 8 connection(s)
I20260812 21:11:32.519606 8442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:32.528322 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.520276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:36569 (local address 127.11.207.190:44453)
0812 21:11:32.520376 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.520378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.520539 (+ 161us) server_negotiation.cc:407] Connection header received
0812 21:11:32.521091 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.521093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.521110 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.521164 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:32.522111 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.522627 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.523606 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.523812 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.523869 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.525936 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:32.525938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:32.525938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:32.525946 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:32.527033 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.527393 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.527394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.527394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.527450 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.527663 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.527664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.527665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.527779 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:32.527837 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.528098 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.528188 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:11:32.528458 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.520244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:36569)
0812 21:11:32.520418 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:32.520461 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:32.520987 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:32.521237 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:32.521241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:32.521818 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:32.522009 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.522015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.522714 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.522715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.523518 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.523520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.523885 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.523886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.524007 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.524496 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:32.524504 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:32.525845 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:32.527106 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:32.527114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:32.527116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:32.527325 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:32.527505 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:32.527506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:32.527507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:32.527607 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:32.527841 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:32.527848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:32.527977 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 21:11:32.528066 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:11:32.528218 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260812 21:11:32.528532 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 8442
I20260812 21:11:32.528638 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:11:32.528853 12094 external_mini_cluster.cc:1428] Setting key ddd07932bdb19a765457b04ed3b48dd5
I20260812 21:11:32.528905 8695 heartbeater.cc:347] Connected to a master server at 127.11.207.190:44453
I20260812 21:11:32.529004 8695 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:32.529177 8695 heartbeater.cc:511] Master 127.11.207.190:44453 requested a full tablet report, sending...
I20260812 21:11:32.529784 8306 ts_manager.cc:194] Registered new tserver with Master: 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865)
I20260812 21:11:32.530596 8306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:36569
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:11:32.554862 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:44453
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:32.636883 8703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:32.637099 8703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:32.637122 8703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:32.638288 8703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:32.638362 8703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:32.638388 8703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:32.639905 8703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:32.640146 8703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:32.640404 8703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:32.645536 8715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:32.645669 8703 server_base.cc:1034] running on GCE node
W20260812 21:11:32.645536 8711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.645514 8710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:32.645958 8703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:32.646186 8703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:32.647334 8703 hybrid_clock.cc:648] HybridClock initialized: now 1786569092647316 us; error 34 us; skew 500 ppm
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:32.649886 8703 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:32.650311 8703 webserver.cc:460] Webserver started at http://127.11.207.130:38075/ using document root <none> and password file <none>
I20260812 21:11:32.650535 8703 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:32.650579 8703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:32.650709 8703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:32.651780 8703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "18828fc1652841d9bb58561a468f495e"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "e6f29ceebc352737b06e0152ee257498"
server_key_iv: "ac0e9cca3b8e10ce04d6b1935ad7d8f2"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.652097 8703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "18828fc1652841d9bb58561a468f495e"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "e6f29ceebc352737b06e0152ee257498"
server_key_iv: "ac0e9cca3b8e10ce04d6b1935ad7d8f2"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.653360 8703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:11:32.654095 8724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.654244 8703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:32.654326 8703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "18828fc1652841d9bb58561a468f495e"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "e6f29ceebc352737b06e0152ee257498"
server_key_iv: "ac0e9cca3b8e10ce04d6b1935ad7d8f2"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.654393 8703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:32.670684 8703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:32.672528 8703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:32.672652 8703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:32.672878 8703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:32.673198 8703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:32.673240 8703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.673271 8703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:32.673293 8703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.678949 8703 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:40559
I20260812 21:11:32.679036 8932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:40559 every 8 connection(s)
I20260812 21:11:32.679323 8703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:32.680471 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 8703
I20260812 21:11:32.680565 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:11:32.680744 12094 external_mini_cluster.cc:1428] Setting key ccd8b6c4961f0d1d9a442b78c40f5eb2
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:32.689347 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.680079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:40227 (local address 127.11.207.190:44453)
0812 21:11:32.680170 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.680171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.680357 (+ 186us) server_negotiation.cc:407] Connection header received
0812 21:11:32.680939 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.680940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.680953 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.681019 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:32.682003 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.682512 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.683510 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.683721 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.683777 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.686253 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:32.686255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:32.686255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:32.686265 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:32.687885 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.688290 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.688291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.688291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.688378 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.688614 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.688615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.688615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.688778 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:32.688852 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.689133 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.689222 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 21:11:32.689435 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.680018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:40227)
0812 21:11:32.680211 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:32.680252 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:32.680839 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:32.681094 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:32.681099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:32.681709 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:32.681911 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.681917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.682597 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.682598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.683427 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.683429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.683792 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.683793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.683932 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.684475 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:32.684486 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:32.686153 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:32.687961 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:32.687969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:32.687971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:32.688201 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:32.688443 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:32.688444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:32.688445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:32.688557 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:32.688838 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:32.688843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:32.689005 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 21:11:32.689094 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:11:32.689231 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260812 21:11:32.690106 8933 heartbeater.cc:347] Connected to a master server at 127.11.207.190:44453
I20260812 21:11:32.690213 8933 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:32.690405 8933 heartbeater.cc:511] Master 127.11.207.190:44453 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
I20260812 21:11:32.690778 8306 ts_manager.cc:194] Registered new tserver with Master: 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559)
I20260812 21:11:32.691157 8306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:40227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:11:32.707579 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:44453
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:32.790766 8949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:32.790988 8949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:32.791020 8949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:32.792182 8949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:32.792224 8949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:32.792259 8949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:32.793766 8949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:32.794023 8949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:32.794247 8949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:32.799122 8964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.799203 8961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:32.799257 8960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:32.799580 8949 server_base.cc:1034] running on GCE node
I20260812 21:11:32.799763 8949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:32.799968 8949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:32.801105 8949 hybrid_clock.cc:648] HybridClock initialized: now 1786569092801089 us; error 27 us; skew 500 ppm
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:32.803470 8949 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:32.803869 8949 webserver.cc:460] Webserver started at http://127.11.207.131:36035/ using document root <none> and password file <none>
I20260812 21:11:32.804082 8949 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:32.804126 8949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:32.804247 8949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:11:32.805325 8949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "ec80641e788f441181390b8638bab15f"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "36ada641079de0cdc34b804cd0e59d36"
server_key_iv: "00e6d6dbc74fc6dab0f997d510c39925"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.805644 8949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "ec80641e788f441181390b8638bab15f"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "36ada641079de0cdc34b804cd0e59d36"
server_key_iv: "00e6d6dbc74fc6dab0f997d510c39925"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.806891 8949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:11:32.807603 8974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.807781 8949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:32.807883 8949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ec80641e788f441181390b8638bab15f"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "36ada641079de0cdc34b804cd0e59d36"
server_key_iv: "00e6d6dbc74fc6dab0f997d510c39925"
server_key_version: "encryptionkey@0"
I20260812 21:11:32.807941 8949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:32.835127 8949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:32.837240 8949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:32.837391 8949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:32.837627 8949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:32.837944 8949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:32.837980 8949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.838011 8949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:32.838033 8949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:32.843973 8949 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:39273
I20260812 21:11:32.844018 9185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:39273 every 8 connection(s)
I20260812 21:11:32.844334 8949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:11:32 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569092, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:32.853035 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.845054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:42193 (local address 127.11.207.190:44453)
0812 21:11:32.845133 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.845135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.845303 (+ 168us) server_negotiation.cc:407] Connection header received
0812 21:11:32.845779 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.845781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.845792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.845841 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:32.846748 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.847235 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.848250 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.848520 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.848578 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.850633 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:32.850634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:32.850634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:32.850642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:32.851811 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.852173 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.852174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.852174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.852230 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.852460 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.852461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.852461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.852566 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:32.852616 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.852870 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.852954 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 21:11:32.853123 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.845023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:42193)
0812 21:11:32.845187 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:32.845224 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:32.845700 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:32.845904 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:32.845907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:32.846476 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:32.846680 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.846685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.847308 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.847309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.848173 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.848176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.848590 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.848591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.848747 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.849202 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:32.849211 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:32.850544 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:32.851889 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:32.851896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:32.851898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:32.852107 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:32.852280 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:32.852281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:32.852281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:32.852416 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:32.852611 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:32.852614 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:32.852729 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 21:11:32.852825 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:11:32.852952 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260812 21:11:32.853420 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 8949
I20260812 21:11:32.853504 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:11:32.853677 12094 external_mini_cluster.cc:1428] Setting key 1c878c6b2db7cae7e961aa66facfb71c
I20260812 21:11:32.853654 9189 heartbeater.cc:347] Connected to a master server at 127.11.207.190:44453
I20260812 21:11:32.853753 9189 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:32.853955 9189 heartbeater.cc:511] Master 127.11.207.190:44453 requested a full tablet report, sending...
I20260812 21:11:32.854349 8305 ts_manager.cc:194] Registered new tserver with Master: ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
I20260812 21:11:32.854746 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:42193
I20260812 21:11:32.865283 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:11:32.872186 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.866590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.11.207.190:44453)
0812 21:11:32.866652 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.866654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.866715 (+ 61us) server_negotiation.cc:407] Connection header received
0812 21:11:32.866783 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.866784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.866796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.866843 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:32.867376 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.867881 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.868624 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.868823 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.868874 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.869969 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:32.869970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:32.869971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:32.869978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:32.871049 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.871395 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.871396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.871397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.871452 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:32.871618 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:32.871618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:32.871619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:32.871804 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:32.871896 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.872018 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.872094 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:11:32.873804 8305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:11:32.875349 8305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:11:32.881759 9206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.877539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37865 (local address 127.0.0.1:45542)
0812 21:11:32.877867 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:32.877907 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:32.877964 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:32.878338 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:32.878339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:32.878342 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:32.878403 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.878406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.879370 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.879371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.880602 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.880605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.881457 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.881458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.881539 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.881554 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:11:32.881629 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 21:11:32.881683 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":64,"threads_started":1}
I20260812 21:11:32.881919 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.877870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.11.207.130:40559)
0812 21:11:32.878055 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.878060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.878110 (+ 50us) server_negotiation.cc:407] Connection header received
0812 21:11:32.878178 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.878181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.878197 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.878269 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:32.878483 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.879614 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.880870 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.881557 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.881622 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.881673 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.881761 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.881801 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260812 21:11:32.881971 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.877749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40559 (local address 127.0.0.1:48462)
0812 21:11:32.878067 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:32.878074 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:32.878117 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:32.878375 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:32.878376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:32.878378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:32.878414 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.878415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.879729 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.879730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.880784 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.880786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.881633 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.881634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.881698 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.881703 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:11:32.881749 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:11:32.881792 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":93,"threads_started":1}
I20260812 21:11:32.882174 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.877659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.11.207.129:37865)
0812 21:11:32.878177 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.878182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.878191 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:11:32.878228 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.878230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.878244 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.878481 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:32.878558 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.879275 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.880670 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.881374 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.881453 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.881509 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.881957 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.882094 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":99,"threads_started":1}
I20260812 21:11:32.882679 8807 tablet_service.cc:1467] Processing CreateTablet for tablet 864c433b37934405903e3b3545bcb1f5 (DEFAULT_TABLE table=test-table [id=545853f24631485bab215f2a72625c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:32.882958 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.877550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:39273 (local address 127.0.0.1:41640)
0812 21:11:32.877976 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:32.877986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:32.878048 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:32.878235 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:32.878238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:32.878240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:32.878319 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.878321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.880864 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.880865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.881583 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:32.881585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.882716 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.882718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:32.882793 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.882797 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:32.882852 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 21:11:32.882894 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":79,"threads_started":1}
I20260812 21:11:32.882946 8563 tablet_service.cc:1467] Processing CreateTablet for tablet 864c433b37934405903e3b3545bcb1f5 (DEFAULT_TABLE table=test-table [id=545853f24631485bab215f2a72625c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:32.883066 8807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 864c433b37934405903e3b3545bcb1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:32.883244 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:32.877722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.11.207.131:39273)
0812 21:11:32.877921 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:32.877926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:32.878040 (+ 114us) server_negotiation.cc:407] Connection header received
0812 21:11:32.878115 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:32.878117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:32.878133 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:32.878221 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:32.878408 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.879622 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.881667 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:32.882548 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:32.882620 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:32.882670 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:32.882932 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 21:11:32.882979 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260812 21:11:32.883277 8563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 864c433b37934405903e3b3545bcb1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:32.884042 9075 tablet_service.cc:1467] Processing CreateTablet for tablet 864c433b37934405903e3b3545bcb1f5 (DEFAULT_TABLE table=test-table [id=545853f24631485bab215f2a72625c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:32.884389 9075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 864c433b37934405903e3b3545bcb1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:32.885818 9214 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap starting.
I20260812 21:11:32.886360 9214 tablet_bootstrap.cc:654] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:32.886360 9215 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Bootstrap starting.
I20260812 21:11:32.886930 9214 log.cc:824] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:32.886982 9215 tablet_bootstrap.cc:654] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:32.887338 9218 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Bootstrap starting.
I20260812 21:11:32.887548 9215 log.cc:824] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:32.887758 9214 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: No bootstrap required, opened a new log
I20260812 21:11:32.887835 9214 ts_tablet_manager.cc:1397] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:32.887979 9218 tablet_bootstrap.cc:654] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:32.888506 9218 log.cc:824] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:32.889215 9215 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: No bootstrap required, opened a new log
I20260812 21:11:32.889245 9218 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: No bootstrap required, opened a new log
I20260812 21:11:32.889290 9215 ts_tablet_manager.cc:1397] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 21:11:32.889294 9218 ts_tablet_manager.cc:1397] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:11:32.889633 9214 raft_consensus.cc:348] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:32.889772 9214 raft_consensus.cc:374] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:32.889806 9214 raft_consensus.cc:729] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665d9eb8786f406f94a5b9245fc02e0d, State: Initialized, Role: FOLLOWER
I20260812 21:11:32.889926 9214 consensus_queue.cc:260] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:32.890194 9214 ts_tablet_manager.cc:1428] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:11:32.890192 8695 heartbeater.cc:503] Master 127.11.207.190:44453 was elected leader, sending a full tablet report...
I20260812 21:11:32.891150 9218 raft_consensus.cc:348] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:32.891145 9215 raft_consensus.cc:348] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:32.891319 9215 raft_consensus.cc:374] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:32.891319 9218 raft_consensus.cc:374] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:32.891364 9218 raft_consensus.cc:729] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec80641e788f441181390b8638bab15f, State: Initialized, Role: FOLLOWER
I20260812 21:11:32.891364 9215 raft_consensus.cc:729] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18828fc1652841d9bb58561a468f495e, State: Initialized, Role: FOLLOWER
I20260812 21:11:32.891471 9215 consensus_queue.cc:260] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:32.891471 9218 consensus_queue.cc:260] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:32.891739 9215 ts_tablet_manager.cc:1428] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:11:32.891762 9189 heartbeater.cc:503] Master 127.11.207.190:44453 was elected leader, sending a full tablet report...
I20260812 21:11:32.891862 8933 heartbeater.cc:503] Master 127.11.207.190:44453 was elected leader, sending a full tablet report...
I20260812 21:11:32.891755 9218 ts_tablet_manager.cc:1428] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 21:11:32.929898 8935 tablet.cc:2369] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:33.020372 8696 tablet.cc:2369] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:11:33.094976 9191 tablet.cc:2369] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:33.165400 9222 raft_consensus.cc:482] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:33.165575 9222 raft_consensus.cc:504] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:33.166010 9222 leader_election.cc:288] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865), 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559)
I20260812 21:11:33.168849 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:33.166262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:46017 (local address 127.11.207.129:37865)
0812 21:11:33.166331 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:33.166333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:33.166343 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:11:33.166377 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:33.166378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:33.166391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:33.166446 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:33.166612 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.167194 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.168098 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.168539 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.168601 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:33.168610 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:33.168764 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 21:11:33.168794 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:11:33.168866 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:33.166185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37865 (local address 127.11.207.131:46017)
0812 21:11:33.166241 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:33.166255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:33.166307 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:33.166478 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:33.166479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:33.166482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:33.166552 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:33.166554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.167277 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.167278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:33.168016 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:33.168018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.168607 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.168608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:33.168694 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:33.168699 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:11:33.168748 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:11:33.168791 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260812 21:11:33.169288 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:33.166304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40559 (local address 127.11.207.131:54571)
0812 21:11:33.166582 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:33.166592 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:33.166654 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:33.166804 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:33.166805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:33.166807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:33.166890 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:33.166892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.167516 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.167517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:33.168286 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:33.168288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.169053 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.169054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:33.169119 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:33.169123 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:33.169184 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:11:33.169217 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":22,"thread_start_us":230,"threads_started":1}
I20260812 21:11:33.169296 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:33.166403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:54571 (local address 127.11.207.130:40559)
0812 21:11:33.166507 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:33.166510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:33.166686 (+ 176us) server_negotiation.cc:407] Connection header received
0812 21:11:33.166722 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:33.166724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:33.166736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:33.166796 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:33.166945 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.167435 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.168379 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.168904 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.168967 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:33.168975 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:33.169209 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 21:11:33.169247 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 21:11:33.169241 8602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c433b37934405903e3b3545bcb1f5" candidate_uuid: "ec80641e788f441181390b8638bab15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "665d9eb8786f406f94a5b9245fc02e0d" is_pre_election: true
I20260812 21:11:33.169415 8602 raft_consensus.cc:2454] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec80641e788f441181390b8638bab15f in term 0.
I20260812 21:11:33.169580 8848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c433b37934405903e3b3545bcb1f5" candidate_uuid: "ec80641e788f441181390b8638bab15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18828fc1652841d9bb58561a468f495e" is_pre_election: true
I20260812 21:11:33.169662 8980 leader_election.cc:302] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d, ec80641e788f441181390b8638bab15f; no voters:
I20260812 21:11:33.169757 8848 raft_consensus.cc:2454] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec80641e788f441181390b8638bab15f in term 0.
I20260812 21:11:33.169781 9222 raft_consensus.cc:2781] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:11:33.169865 9222 raft_consensus.cc:482] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:11:33.169893 9222 raft_consensus.cc:3037] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:33.170665 9222 raft_consensus.cc:504] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:33.170781 9222 leader_election.cc:288] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [CANDIDATE]: Term 1 election: Requested vote from peers 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865), 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559)
I20260812 21:11:33.170986 8602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c433b37934405903e3b3545bcb1f5" candidate_uuid: "ec80641e788f441181390b8638bab15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "665d9eb8786f406f94a5b9245fc02e0d"
I20260812 21:11:33.171000 8848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c433b37934405903e3b3545bcb1f5" candidate_uuid: "ec80641e788f441181390b8638bab15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18828fc1652841d9bb58561a468f495e"
I20260812 21:11:33.171070 8602 raft_consensus.cc:3037] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:33.171072 8848 raft_consensus.cc:3037] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:33.171777 8848 raft_consensus.cc:2454] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec80641e788f441181390b8638bab15f in term 1.
I20260812 21:11:33.171797 8602 raft_consensus.cc:2454] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec80641e788f441181390b8638bab15f in term 1.
I20260812 21:11:33.171940 8980 leader_election.cc:302] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f; no voters:
I20260812 21:11:33.172037 9222 raft_consensus.cc:2781] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:33.172199 9222 raft_consensus.cc:686] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 1 LEADER]: Becoming Leader. State: Replica: ec80641e788f441181390b8638bab15f, State: Running, Role: LEADER
I20260812 21:11:33.172278 9222 consensus_queue.cc:237] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:33.173136 8305 catalog_manager.cc:5495] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f reported cstate change: term changed from 0 to 1, leader changed from <none> to ec80641e788f441181390b8638bab15f (127.11.207.131). New cstate: current_term: 1 leader_uuid: "ec80641e788f441181390b8638bab15f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } health_report { overall_health: UNKNOWN } } }
I20260812 21:11:33.240008 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:33.237641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.11.207.131:39273)
0812 21:11:33.237756 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:33.237758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:33.237767 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:11:33.237806 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:33.237807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:33.237828 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:33.237890 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:33.238032 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.238510 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.239251 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:33.239447 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:33.239501 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:33.239674 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:33.239783 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:11:33.239855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:33.239878 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 21:11:33.239900 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 21:11:33.241959 8602 raft_consensus.cc:1261] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Refusing update from remote peer ec80641e788f441181390b8638bab15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:33.241959 8848 raft_consensus.cc:1261] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Refusing update from remote peer ec80641e788f441181390b8638bab15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:33.242327 9226 consensus_queue.cc:1035] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:33.242488 9222 consensus_queue.cc:1035] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:33.244825 9229 mvcc.cc:205] Tried to move back new op lower bound from 7317787005915951104 to 7317787005634310144. Current Snapshot: MvccSnapshot[applied={T|T < 7317787005915951104}]
I20260812 21:11:33.245690 9230 mvcc.cc:205] Tried to move back new op lower bound from 7317787005915951104 to 7317787005634310144. Current Snapshot: MvccSnapshot[applied={T|T < 7317787005915951104}]
I20260812 21:11:33.247143 8305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:33.247241 8305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:33.248021 8305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Sending DeleteTablet for 3 replicas of tablet 864c433b37934405903e3b3545bcb1f5
I20260812 21:11:33.248358 9075 tablet_service.cc:1514] Processing DeleteTablet for tablet 864c433b37934405903e3b3545bcb1f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:33 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:41640
I20260812 21:11:33.248396 8563 tablet_service.cc:1514] Processing DeleteTablet for tablet 864c433b37934405903e3b3545bcb1f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:33 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:45542
I20260812 21:11:33.248409 8807 tablet_service.cc:1514] Processing DeleteTablet for tablet 864c433b37934405903e3b3545bcb1f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:33 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:48462
I20260812 21:11:33.248615 9243 tablet_replica.cc:330] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: stopping tablet replica
I20260812 21:11:33.248752 9243 raft_consensus.cc:2229] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:11:33.248800 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 8442
I20260812 21:11:33.248852 9243 raft_consensus.cc:2258] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:33.255571 9243 ts_tablet_manager.cc:1905] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:33.257383 9243 ts_tablet_manager.cc:1918] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:33.257510 9243 log.cc:1196] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/864c433b37934405903e3b3545bcb1f5
I20260812 21:11:33.257817 9243 ts_tablet_manager.cc:1939] T 864c433b37934405903e3b3545bcb1f5 P ec80641e788f441181390b8638bab15f: Deleting consensus metadata
I20260812 21:11:33.258345 8282 catalog_manager.cc:4841] TS ec80641e788f441181390b8638bab15f (127.11.207.131:39273): tablet 864c433b37934405903e3b3545bcb1f5 (table test-table [id=545853f24631485bab215f2a72625c5e]) successfully deleted
I20260812 21:11:33.260457 9246 tablet_replica.cc:330] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: stopping tablet replica
I20260812 21:11:33.260618 9246 raft_consensus.cc:2229] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:33.260721 9246 raft_consensus.cc:2258] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:33.262614 9246 ts_tablet_manager.cc:1905] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:33.264214 9246 ts_tablet_manager.cc:1918] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:33.264302 9246 log.cc:1196] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/864c433b37934405903e3b3545bcb1f5
I20260812 21:11:33.264631 9246 ts_tablet_manager.cc:1939] T 864c433b37934405903e3b3545bcb1f5 P 18828fc1652841d9bb58561a468f495e: Deleting consensus metadata
I20260812 21:11:33.265239 8283 catalog_manager.cc:4841] TS 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): tablet 864c433b37934405903e3b3545bcb1f5 (table test-table [id=545853f24631485bab215f2a72625c5e]) successfully deleted
W20260812 21:11:33.270583 8283 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:37865 (error 108)
I20260812 21:11:33.270818 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 8703
I20260812 21:11:33.281957 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:33.281786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37865 (local address 127.0.0.1:45556)
0812 21:11:33.281853 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:33.281908 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 21:11:33.282146 8283 catalog_manager.cc:4584] TS 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 864c433b37934405903e3b3545bcb1f5: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37865: connect: Connection refused (error 111)
I20260812 21:11:33.296494 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 8949
I20260812 21:11:33.312969 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 8248
I20260812 21:11:53.328637 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44453
--webserver_interface=127.11.207.190
--webserver_port=33633
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:44453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:11:53.412011 9258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:53.412222 9258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:53.412243 9258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:53.413450 9258 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 21:11:53.413496 9258 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 21:11:53.413508 9258 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 21:11:53.413518 9258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:11:53.413533 9258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:53.413558 9258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:53.413581 9258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:11:53.413601 9258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:11:53.415125 9258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:44453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=33633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:53.415361 9258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:53.415598 9258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:53.425029 9275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.425050 9272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.425019 9270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:53.425064 9258 server_base.cc:1034] running on GCE node
I20260812 21:11:53.425562 9258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:53.425923 9258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:53.427066 9258 hybrid_clock.cc:648] HybridClock initialized: now 1786569113427049 us; error 34 us; skew 500 ppm
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:53.429641 9258 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:11:53.430116 9258 webserver.cc:460] Webserver started at http://127.11.207.190:33633/ using document root <none> and password file <none>
I20260812 21:11:53.430351 9258 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:53.430414 9258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:53.431807 9258 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:53.432474 9287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.432614 9258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:11:53.432708 9258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "ae5366270f46483398ab62d4e51646df"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "26d5b65e5560bb2f5934746b0767857c"
server_key_iv: "d1a18ea639cc8e9603449338ee8ea4b5"
server_key_version: "encryptionkey@0"
I20260812 21:11:53.433131 9258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:53.448555 9258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:53.450409 9258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:53.450551 9258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:53.454916 9258 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:44453
I20260812 21:11:53.454964 9391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:44453 every 8 connection(s)
I20260812 21:11:53.455354 9258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:11:53.456899 9392 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:11:53.457541 9392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap starting.
I20260812 21:11:53.464967 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 9258
I20260812 21:11:53.465317 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:37865
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:44453
--webserver_port=45607
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:53.467250 9392 log.cc:824] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:53.469383 9392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:53.469672 9392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap complete.
I20260812 21:11:53.472038 9392 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:53.472275 9392 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5366270f46483398ab62d4e51646df, State: Initialized, Role: FOLLOWER
I20260812 21:11:53.472457 9392 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:53.472534 9392 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:11:53.472571 9392 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:11:53.472625 9392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:11:53.473665 9392 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:53.473757 9392 leader_election.cc:302] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae5366270f46483398ab62d4e51646df; no voters:
I20260812 21:11:53.473919 9392 leader_election.cc:288] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:11:53.473973 9401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:11:53.474136 9401 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 LEADER]: Becoming Leader. State: Replica: ae5366270f46483398ab62d4e51646df, State: Running, Role: LEADER
I20260812 21:11:53.474208 9392 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:11:53.474256 9401 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:11:53.474575 9402 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ae5366270f46483398ab62d4e51646df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } } }
I20260812 21:11:53.474575 9401 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae5366270f46483398ab62d4e51646df. Latest consensus state: current_term: 2 leader_uuid: "ae5366270f46483398ab62d4e51646df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } } }
I20260812 21:11:53.474766 9402 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:53.474776 9401 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: This master's current role is: LEADER
I20260812 21:11:53.475124 9414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:11:53.476015 9414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:11:53.476181 9414 catalog_manager.cc:1259] Loaded cluster ID: 449d34439dd94aadaf7c267c00e48144
I20260812 21:11:53.476473 9414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:11:53.477291 9414 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:11:53.477479 9414 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Loaded TSK: 0
I20260812 21:11:53.483549 9414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Generated new TSK 1
I20260812 21:11:53.483635 9414 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:11:53.568207 9398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:53.568459 9398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:53.568495 9398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:53.569655 9398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:53.569698 9398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:53.569732 9398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:11:53.571206 9398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:37865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=45607
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:53.571460 9398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:53.571697 9398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:53.576810 9439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.576826 9436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.576810 9435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:53.577385 9398 server_base.cc:1034] running on GCE node
I20260812 21:11:53.577581 9398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:53.577803 9398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:53.578969 9398 hybrid_clock.cc:648] HybridClock initialized: now 1786569113578956 us; error 28 us; skew 500 ppm
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:53.581619 9398 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:11:53.582083 9398 webserver.cc:460] Webserver started at http://127.11.207.129:45607/ using document root <none> and password file <none>
I20260812 21:11:53.582330 9398 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:53.582386 9398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:53.583742 9398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:53.584446 9452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.584594 9398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:53.584676 9398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "665d9eb8786f406f94a5b9245fc02e0d"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "f7fa5318979bb05c7e7d9a64f99ea7ff"
server_key_iv: "6bf370eb9cfb765e34c2618fdccbcd06"
server_key_version: "encryptionkey@0"
I20260812 21:11:53.585016 9398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:53.597288 9398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:53.599350 9398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:53.599493 9398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:53.599751 9398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:53.600302 9461 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:11:53.602263 9398 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:11:53.602308 9398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:11:53.602355 9398 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:11:53.602983 9398 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:11:53.603017 9398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:53.603111 9461 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap starting.
I20260812 21:11:53.610033 9398 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:37865
I20260812 21:11:53.610194 9665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:37865 every 8 connection(s)
I20260812 21:11:53.610474 9398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:11:53.611898 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 9398
I20260812 21:11:53.612154 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:40559
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:44453
--webserver_port=38075
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:11:53.615146 9461 log.cc:824] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:53.618625 9461 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:11:53.618965 9461 tablet_bootstrap.cc:492] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap complete.
I20260812 21:11:53.620209 9461 ts_tablet_manager.cc:1397] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.007s
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:53.621584 9461 raft_consensus.cc:348] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:53.621932 9461 raft_consensus.cc:729] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665d9eb8786f406f94a5b9245fc02e0d, State: Initialized, Role: FOLLOWER
I20260812 21:11:53.622076 9461 consensus_queue.cc:260] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } }
I20260812 21:11:53.622363 9461 ts_tablet_manager.cc:1428] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent starting tablet: real 0.002s user 0.005s sys 0.001s
I20260812 21:11:53.625741 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.611290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:33289 (local address 127.11.207.190:44453)
0812 21:11:53.611481 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.611491 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.611715 (+ 224us) server_negotiation.cc:407] Connection header received
0812 21:11:53.614680 (+ 2965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.614687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.614751 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.614900 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:53.615770 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.616964 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.618453 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.619195 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.619314 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.622513 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:53.622518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:53.622518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:53.622530 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:53.624043 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.624478 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.624482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.624482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.624589 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.624831 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.624831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.624832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.625053 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:53.625126 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.625422 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.625553 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260812 21:11:53.625741 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.611185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:33289)
0812 21:11:53.611567 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:53.611602 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:53.614354 (+ 2752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:53.615053 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:53.615058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:53.615368 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:53.615654 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.615662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.617119 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.617120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.618351 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.618354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.619440 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.619441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.619584 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.620194 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:53.620205 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:53.622338 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:53.624126 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:53.624133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:53.624135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:53.624383 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:53.624651 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:53.624652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:53.624652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:53.624753 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:53.625119 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:53.625126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:53.625251 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 21:11:53.625339 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:11:53.625515 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":97,"threads_started":1}
I20260812 21:11:53.626748 9667 heartbeater.cc:347] Connected to a master server at 127.11.207.190:44453
I20260812 21:11:53.626849 9667 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:53.627022 9667 heartbeater.cc:511] Master 127.11.207.190:44453 requested a full tablet report, sending...
I20260812 21:11:53.627694 9320 ts_manager.cc:194] Registered new tserver with Master: 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865)
I20260812 21:11:53.628763 9320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:33289
I20260812 21:11:53.629578 9667 heartbeater.cc:503] Master 127.11.207.190:44453 was elected leader, sending a full tablet report...
I20260812 21:11:53.633579 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.628550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.11.207.129:37865)
0812 21:11:53.628704 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.628708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.630388 (+ 1680us) server_negotiation.cc:407] Connection header received
0812 21:11:53.630479 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.630481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.630494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.630550 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:53.630678 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.631329 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.632780 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.633234 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.633293 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.633330 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.633495 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.633526 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260812 21:11:53.633611 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.628264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37865 (local address 127.0.0.1:60840)
0812 21:11:53.630271 (+ 2007us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:53.630309 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:53.630394 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:53.630561 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:53.630564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:53.630565 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:53.630626 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.630627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.631403 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.631404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.632700 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.632704 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.633307 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.633307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.633397 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.633412 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:11:53.633475 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 21:11:53.633531 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1979,"thread_start_us":123,"threads_started":1}
I20260812 21:11:53.634045 9542 tablet_service.cc:1514] Processing DeleteTablet for tablet 864c433b37934405903e3b3545bcb1f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:33 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:60840
I20260812 21:11:53.634536 9683 tablet_replica.cc:330] stopping tablet replica
I20260812 21:11:53.634636 9683 raft_consensus.cc:2229] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:53.634708 9683 raft_consensus.cc:2258] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:53.635004 9683 ts_tablet_manager.cc:1905] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:53.636965 9683 ts_tablet_manager.cc:1918] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:53.637035 9683 log.cc:1196] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal/wals/864c433b37934405903e3b3545bcb1f5
I20260812 21:11:53.637238 9683 ts_tablet_manager.cc:1939] T 864c433b37934405903e3b3545bcb1f5 P 665d9eb8786f406f94a5b9245fc02e0d: Deleting consensus metadata
I20260812 21:11:53.637707 9293 catalog_manager.cc:4841] TS 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865): tablet 864c433b37934405903e3b3545bcb1f5 (table test-table [id=545853f24631485bab215f2a72625c5e]) successfully deleted
W20260812 21:11:53.701354 9678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:53.701560 9678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:53.701592 9678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:53.702716 9678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:53.702759 9678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:53.702792 9678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:11:53.704238 9678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:40559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=38075
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:53.704527 9678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:53.704748 9678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:53.709679 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.709728 9700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.709779 9703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:53.709939 9678 server_base.cc:1034] running on GCE node
I20260812 21:11:53.710209 9678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:53.710443 9678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:53.711586 9678 hybrid_clock.cc:648] HybridClock initialized: now 1786569113711574 us; error 30 us; skew 500 ppm
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:53.714099 9678 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:11:53.714512 9678 webserver.cc:460] Webserver started at http://127.11.207.130:38075/ using document root <none> and password file <none>
I20260812 21:11:53.714709 9678 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:53.714762 9678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:53.716087 9678 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:11:53.716727 9720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.716898 9678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.716989 9678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "18828fc1652841d9bb58561a468f495e"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "e6f29ceebc352737b06e0152ee257498"
server_key_iv: "ac0e9cca3b8e10ce04d6b1935ad7d8f2"
server_key_version: "encryptionkey@0"
I20260812 21:11:53.717316 9678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:53.729941 9678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:53.731916 9678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:53.732028 9678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:53.732213 9678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:53.732522 9678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:53.732558 9678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.732579 9678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:53.732595 9678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.738256 9678 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:40559
I20260812 21:11:53.738334 9932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:40559 every 8 connection(s)
I20260812 21:11:53.738667 9678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:11:53.738906 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 9678
I20260812 21:11:53.739200 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:39273
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:44453
--webserver_port=36035
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:53.749083 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.739723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:46479 (local address 127.11.207.190:44453)
0812 21:11:53.739767 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.739769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.739781 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:11:53.740433 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.740435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.740446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.740487 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:53.741604 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.742322 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.743657 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.743853 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.743898 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.746278 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:53.746280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:53.746280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:53.746290 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:53.747713 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.748076 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.748077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.748078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.748146 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.748408 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.748409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.748409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.748558 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:53.748635 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.748885 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.748986 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:11:53.749111 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.739469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:46479)
0812 21:11:53.739643 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:53.739694 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:53.740296 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:53.740561 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:53.740565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:53.741300 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:53.741507 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.741513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.742415 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.742416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.743581 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.743584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.743938 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.743939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.744100 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.744646 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:53.744658 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:53.746183 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:53.747791 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:53.747798 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:53.747800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:53.748006 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:53.748211 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:53.748211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:53.748211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:53.748311 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:53.748622 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:53.748626 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:53.748746 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 21:11:53.748821 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 21:11:53.748944 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260812 21:11:53.749678 9933 heartbeater.cc:347] Connected to a master server at 127.11.207.190:44453
I20260812 21:11:53.749797 9933 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:53.750026 9933 heartbeater.cc:511] Master 127.11.207.190:44453 requested a full tablet report, sending...
I20260812 21:11:53.750409 9320 ts_manager.cc:194] Registered new tserver with Master: 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559)
I20260812 21:11:53.750769 9320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:46479
W20260812 21:11:53.829542 9939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:11:53.829787 9939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:11:53.829813 9939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:11:53.831022 9939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:11:53.831065 9939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:11:53.831095 9939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:11:53.832653 9939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:39273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=36035
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:11:53.832952 9939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:11:53.833171 9939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:11:53.838371 9946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:11:53.838397 9955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:53.838481 9939 server_base.cc:1034] running on GCE node
W20260812 21:11:53.838402 9953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:11:53.838792 9939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:11:53.839049 9939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:11:53.840191 9939 hybrid_clock.cc:648] HybridClock initialized: now 1786569113840177 us; error 32 us; skew 500 ppm
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:11:53.842654 9939 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:11:53.843093 9939 webserver.cc:460] Webserver started at http://127.11.207.131:36035/ using document root <none> and password file <none>
I20260812 21:11:53.843297 9939 fs_manager.cc:359] Metadata directory not provided
I20260812 21:11:53.843348 9939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:11:53.844712 9939 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:11:53.845274 9974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.845428 9939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.845508 9939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ec80641e788f441181390b8638bab15f"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "36ada641079de0cdc34b804cd0e59d36"
server_key_iv: "00e6d6dbc74fc6dab0f997d510c39925"
server_key_version: "encryptionkey@0"
I20260812 21:11:53.845865 9939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:11:53.856933 9939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:11:53.858685 9939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:11:53.858809 9939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:11:53.859050 9939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:11:53.859357 9939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:11:53.859391 9939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.859416 9939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:11:53.859439 9939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.865121 9939 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:39273
I20260812 21:11:53.865207 10193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:39273 every 8 connection(s)
I20260812 21:11:53.865502 9939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:11:53.870415 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 9939
Aug 12 21:11:53 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:11:53.875139 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.866222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:59947 (local address 127.11.207.190:44453)
0812 21:11:53.866279 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.866281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.866492 (+ 211us) server_negotiation.cc:407] Connection header received
0812 21:11:53.866961 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.866963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.866974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.867022 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:53.867939 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.868484 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.869466 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.869665 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.869719 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.871925 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:53.871926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:53.871927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:53.871933 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:53.873339 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.874012 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.874014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.874014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.874075 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.874402 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.874404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.874404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.874560 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:53.874613 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.874985 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.875066 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:11:53.875313 10200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.866227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:59947)
0812 21:11:53.866371 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:53.866408 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:53.866885 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:53.867080 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:53.867083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:53.867670 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:11:53.867870 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.867876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.868572 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.868573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.869384 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.869387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.869726 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.869727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.869827 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.870267 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:11:53.870276 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:11:53.871840 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:11:53.873418 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:53.873425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:53.873427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:53.873748 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:53.874141 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:11:53.874142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:11:53.874142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:11:53.874308 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:11:53.874621 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:11:53.874629 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:11:53.874847 (+ 218us) client_negotiation.cc:769] Sending connection context
0812 21:11:53.874944 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 21:11:53.875079 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260812 21:11:53.875789 10194 heartbeater.cc:347] Connected to a master server at 127.11.207.190:44453
I20260812 21:11:53.875893 10194 heartbeater.cc:464] Registering TS with master...
I20260812 21:11:53.876080 10194 heartbeater.cc:511] Master 127.11.207.190:44453 requested a full tablet report, sending...
I20260812 21:11:53.876525 9320 ts_manager.cc:194] Registered new tserver with Master: ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
I20260812 21:11:53.877003 9320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:59947
I20260812 21:11:53.877420 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.870808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.11.207.190:44453)
0812 21:11:53.870946 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.870948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.870980 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:11:53.871057 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.871057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.871071 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.871130 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:53.871709 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.872279 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.873158 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.873479 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.873529 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.874837 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:53.874838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:53.874839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:53.874844 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:53.876396 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.876703 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.876704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.876704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.876765 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.876917 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.876918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.876918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.877021 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:53.877203 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.877243 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.877336 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260812 21:11:53.878065 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:11:53.880386 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.878382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.11.207.190:44453)
0812 21:11:53.878419 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.878420 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.878506 (+ 86us) server_negotiation.cc:407] Connection header received
0812 21:11:53.878562 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.878563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.878592 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.878643 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:53.878714 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.879154 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.879816 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.879996 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.880038 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.880138 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:53.880179 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 21:11:53.880301 (+ 122us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52564: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:11:53.880443 10204 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52564: token expired
W20260812 21:11:53.880623 9197 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.207.190:44453)
I20260812 21:11:53.885854 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.880846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.11.207.190:44453)
0812 21:11:53.880965 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.880966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.880972 (+ 6us) server_negotiation.cc:407] Connection header received
0812 21:11:53.880987 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.880988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.880995 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.881036 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:11:53.881460 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.881880 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.882475 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.882633 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.882670 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.883680 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:11:53.883681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:11:53.883681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:11:53.883686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:11:53.884901 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.885232 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.885232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.885233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.885281 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:11:53.885473 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:11:53.885473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:11:53.885474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:11:53.885594 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:11:53.885665 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.885734 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.885793 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 21:11:53.887025 9320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:11:53.888752 9320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:11:53.891114 9542 tablet_service.cc:1467] Processing CreateTablet for tablet f7f11b668f074d48af265b86c163d708 (DEFAULT_TABLE table=test-table [id=ad16cc4dc5f64f0f9e27038ee78c0e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:53.891301 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f11b668f074d48af265b86c163d708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:53.892834 9461 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap starting.
I20260812 21:11:53.893141 9461 tablet_bootstrap.cc:654] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:53.894027 9461 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: No bootstrap required, opened a new log
I20260812 21:11:53.894086 9461 ts_tablet_manager.cc:1397] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 21:11:53.894210 9461 raft_consensus.cc:348] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:53.894272 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.890531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40559 (local address 127.0.0.1:49582)
0812 21:11:53.890668 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:53.890680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:53.890718 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:53.890973 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:53.890973 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:53.890975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:53.891018 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.891020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.892083 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.892084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.893018 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.893021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.894054 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.894055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.894118 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.894122 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:11:53.894176 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:11:53.894222 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 21:11:53.894290 9461 raft_consensus.cc:374] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:53.894345 9461 raft_consensus.cc:729] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665d9eb8786f406f94a5b9245fc02e0d, State: Initialized, Role: FOLLOWER
I20260812 21:11:53.894369 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.890648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.11.207.130:40559)
0812 21:11:53.890829 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.890834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.890842 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:11:53.890887 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.890889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.890901 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.890969 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:53.891079 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.892014 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.893106 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.893977 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.894058 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.894127 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.894238 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.894285 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260812 21:11:53.894395 9461 consensus_queue.cc:260] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:53.894490 9461 ts_tablet_manager.cc:1428] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:11:53.895051 9808 tablet_service.cc:1467] Processing CreateTablet for tablet f7f11b668f074d48af265b86c163d708 (DEFAULT_TABLE table=test-table [id=ad16cc4dc5f64f0f9e27038ee78c0e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:53.895434 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f11b668f074d48af265b86c163d708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:53.896517 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.890564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:39273 (local address 127.0.0.1:51406)
0812 21:11:53.892455 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:53.892466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:53.892527 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:53.892706 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:53.892707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:53.892709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:53.892751 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.892753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.893770 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.893771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.895476 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:53.895479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.896254 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.896255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:53.896317 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.896329 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 21:11:53.896407 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 21:11:53.896448 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1871,"thread_start_us":102,"threads_started":1}
I20260812 21:11:53.896545 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:53.890829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.11.207.131:39273)
0812 21:11:53.890984 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:53.890989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:53.892541 (+ 1552us) server_negotiation.cc:407] Connection header received
0812 21:11:53.892597 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:53.892600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:53.892617 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:53.892699 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:53.892810 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.893696 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.895549 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:53.896183 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:53.896246 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:53.896298 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:53.896422 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 21:11:53.896463 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260812 21:11:53.897128 10060 tablet_service.cc:1467] Processing CreateTablet for tablet f7f11b668f074d48af265b86c163d708 (DEFAULT_TABLE table=test-table [id=ad16cc4dc5f64f0f9e27038ee78c0e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:11:53.897393 10060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7f11b668f074d48af265b86c163d708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:11:53.898358 10211 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Bootstrap starting.
I20260812 21:11:53.899044 10211 tablet_bootstrap.cc:654] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:53.899508 10211 log.cc:824] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:53.900259 10211 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: No bootstrap required, opened a new log
I20260812 21:11:53.900285 10213 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Bootstrap starting.
I20260812 21:11:53.900331 10211 ts_tablet_manager.cc:1397] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:11:53.900966 10213 tablet_bootstrap.cc:654] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Neither blocks nor log segments found. Creating new log.
I20260812 21:11:53.901561 10213 log.cc:824] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:11:53.902307 10213 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: No bootstrap required, opened a new log
I20260812 21:11:53.902386 10213 ts_tablet_manager.cc:1397] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:11:53.902266 10211 raft_consensus.cc:348] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:53.902431 10211 raft_consensus.cc:374] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:53.902478 10211 raft_consensus.cc:729] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18828fc1652841d9bb58561a468f495e, State: Initialized, Role: FOLLOWER
I20260812 21:11:53.902595 10211 consensus_queue.cc:260] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:53.902889 10211 ts_tablet_manager.cc:1428] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:11:53.902885 9933 heartbeater.cc:503] Master 127.11.207.190:44453 was elected leader, sending a full tablet report...
I20260812 21:11:53.904129 10213 raft_consensus.cc:348] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:53.904258 10213 raft_consensus.cc:374] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:11:53.904279 10213 raft_consensus.cc:729] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec80641e788f441181390b8638bab15f, State: Initialized, Role: FOLLOWER
I20260812 21:11:53.904430 10213 consensus_queue.cc:260] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:53.904675 10213 ts_tablet_manager.cc:1428] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:11:53.904695 10194 heartbeater.cc:503] Master 127.11.207.190:44453 was elected leader, sending a full tablet report...
W20260812 21:11:53.989264 9935 tablet.cc:2369] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:11:54.023456 10220 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:11:54.023603 10220 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:54.023962 10220 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec80641e788f441181390b8638bab15f (127.11.207.131:39273), 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865)
I20260812 21:11:54.026690 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:54.024277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:43327 (local address 127.11.207.129:37865)
0812 21:11:54.024328 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:54.024330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:54.024369 (+ 39us) server_negotiation.cc:407] Connection header received
0812 21:11:54.024399 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:54.024399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:54.024409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:54.024451 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:54.024592 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.025076 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.026004 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.026385 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.026447 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:54.026455 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:54.026585 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 21:11:54.026623 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:11:54.026690 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:54.024175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37865 (local address 127.11.207.130:43327)
0812 21:11:54.024234 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:54.024248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:54.024297 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:54.024466 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:54.024467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:54.024468 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:54.024534 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:54.024536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.025149 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.025150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:54.025916 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:54.025918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.026457 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.026458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:54.026526 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:54.026529 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:11:54.026574 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:11:54.026611 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 21:11:54.026849 10222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:54.024187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:39273 (local address 127.11.207.130:51909)
0812 21:11:54.024383 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:54.024394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:11:54.024463 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:11:54.024590 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:11:54.024591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:11:54.024592 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:11:54.024669 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:54.024671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.025274 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.025276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:54.026040 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:11:54.026042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.026599 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.026600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:11:54.026678 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:54.026681 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:11:54.026727 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:11:54.026765 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260812 21:11:54.026847 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:54.024325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:51909 (local address 127.11.207.131:39273)
0812 21:11:54.024388 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:54.024390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:54.024463 (+ 73us) server_negotiation.cc:407] Connection header received
0812 21:11:54.024521 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:54.024522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:54.024531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:54.024580 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:11:54.024731 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.025199 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.026135 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.026521 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.026587 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:54.026595 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:54.026743 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 21:11:54.026784 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:11:54.027140 10105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f11b668f074d48af265b86c163d708" candidate_uuid: "18828fc1652841d9bb58561a468f495e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec80641e788f441181390b8638bab15f" is_pre_election: true
I20260812 21:11:54.027194 9584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f11b668f074d48af265b86c163d708" candidate_uuid: "18828fc1652841d9bb58561a468f495e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "665d9eb8786f406f94a5b9245fc02e0d" is_pre_election: true
I20260812 21:11:54.027323 10105 raft_consensus.cc:2454] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18828fc1652841d9bb58561a468f495e in term 0.
I20260812 21:11:54.027334 9584 raft_consensus.cc:2454] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18828fc1652841d9bb58561a468f495e in term 0.
I20260812 21:11:54.027570 9722 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f; no voters:
I20260812 21:11:54.027714 10220 raft_consensus.cc:2781] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:11:54.027783 10220 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:11:54.027799 10220 raft_consensus.cc:3037] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:54.028542 10220 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:54.028663 10220 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [CANDIDATE]: Term 1 election: Requested vote from peers ec80641e788f441181390b8638bab15f (127.11.207.131:39273), 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865)
I20260812 21:11:54.028846 9584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f11b668f074d48af265b86c163d708" candidate_uuid: "18828fc1652841d9bb58561a468f495e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "665d9eb8786f406f94a5b9245fc02e0d"
I20260812 21:11:54.028882 10105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f11b668f074d48af265b86c163d708" candidate_uuid: "18828fc1652841d9bb58561a468f495e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec80641e788f441181390b8638bab15f"
I20260812 21:11:54.028932 9584 raft_consensus.cc:3037] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:54.028944 10105 raft_consensus.cc:3037] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:11:54.029627 10105 raft_consensus.cc:2454] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18828fc1652841d9bb58561a468f495e in term 1.
I20260812 21:11:54.029700 9584 raft_consensus.cc:2454] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18828fc1652841d9bb58561a468f495e in term 1.
I20260812 21:11:54.029784 9722 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f; no voters:
I20260812 21:11:54.029877 10220 raft_consensus.cc:2781] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:11:54.030025 10220 raft_consensus.cc:686] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 1 LEADER]: Becoming Leader. State: Replica: 18828fc1652841d9bb58561a468f495e, State: Running, Role: LEADER
I20260812 21:11:54.030109 10220 consensus_queue.cc:237] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:11:54.030970 9320 catalog_manager.cc:5495] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e reported cstate change: term changed from 0 to 1, leader changed from <none> to 18828fc1652841d9bb58561a468f495e (127.11.207.130). New cstate: current_term: 1 leader_uuid: "18828fc1652841d9bb58561a468f495e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } health_report { overall_health: UNKNOWN } } }
I20260812 21:11:54.038177 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:54.036149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.11.207.130:40559)
0812 21:11:54.036193 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:11:54.036195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:11:54.036205 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:11:54.036259 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:11:54.036260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:11:54.036275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:11:54.036331 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:11:54.036452 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.036932 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.037580 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:11:54.037782 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:11:54.037829 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:11:54.037935 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:11:54.038026 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:11:54.038077 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:11:54.038089 (+ 12us) server_negotiation.cc:299] Negotiation successful
0812 21:11:54.038104 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:11:54.039805 10105 raft_consensus.cc:1261] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Refusing update from remote peer 18828fc1652841d9bb58561a468f495e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:54.040143 9584 raft_consensus.cc:1261] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Refusing update from remote peer 18828fc1652841d9bb58561a468f495e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:11:54.040139 10223 consensus_queue.cc:1035] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:54.040427 10223 consensus_queue.cc:1035] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [LEADER]: Connected to new peer: Peer: permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:11:54.042644 10236 mvcc.cc:205] Tried to move back new op lower bound from 7317787091104665600 to 7317787091067867136. Current Snapshot: MvccSnapshot[applied={T|T < 7317787091104665600}]
I20260812 21:11:54.043035 10229 mvcc.cc:205] Tried to move back new op lower bound from 7317787091104665600 to 7317787091067867136. Current Snapshot: MvccSnapshot[applied={T|T < 7317787091104665600}]
I20260812 21:11:54.045011 9320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:54.045117 9320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:11:54.045795 9320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Sending DeleteTablet for 3 replicas of tablet f7f11b668f074d48af265b86c163d708
I20260812 21:11:54.046119 9542 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f11b668f074d48af265b86c163d708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:54 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:60840
I20260812 21:11:54.046126 9808 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f11b668f074d48af265b86c163d708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:54 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:49582
I20260812 21:11:54.046119 10060 tablet_service.cc:1514] Processing DeleteTablet for tablet f7f11b668f074d48af265b86c163d708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:11:54 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51406
I20260812 21:11:54.046308 9683 tablet_replica.cc:330] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: stopping tablet replica
I20260812 21:11:54.046372 9683 raft_consensus.cc:2229] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:54.046419 9683 raft_consensus.cc:2258] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:54.046538 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 9398
I20260812 21:11:54.053807 10248 tablet_replica.cc:330] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: stopping tablet replica
I20260812 21:11:54.053956 10248 raft_consensus.cc:2229] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:11:54.054075 10248 raft_consensus.cc:2258] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:54.054265 10244 tablet_replica.cc:330] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: stopping tablet replica
I20260812 21:11:54.054368 10244 raft_consensus.cc:2229] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:11:54.054499 10244 raft_consensus.cc:2258] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:11:54.054768 10248 ts_tablet_manager.cc:1905] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:54.054991 10244 ts_tablet_manager.cc:1905] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:11:54.056965 10248 ts_tablet_manager.cc:1918] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:54.057054 10248 log.cc:1196] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/f7f11b668f074d48af265b86c163d708
I20260812 21:11:54.057405 10248 ts_tablet_manager.cc:1939] T f7f11b668f074d48af265b86c163d708 P ec80641e788f441181390b8638bab15f: Deleting consensus metadata
I20260812 21:11:54.060336 9291 catalog_manager.cc:4841] TS ec80641e788f441181390b8638bab15f (127.11.207.131:39273): tablet f7f11b668f074d48af265b86c163d708 (table test-table [id=ad16cc4dc5f64f0f9e27038ee78c0e82]) successfully deleted
I20260812 21:11:54.063493 10244 ts_tablet_manager.cc:1918] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:11:54.063577 10244 log.cc:1196] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/f7f11b668f074d48af265b86c163d708
I20260812 21:11:54.063844 10244 ts_tablet_manager.cc:1939] T f7f11b668f074d48af265b86c163d708 P 18828fc1652841d9bb58561a468f495e: Deleting consensus metadata
I20260812 21:11:54.064358 9293 catalog_manager.cc:4841] TS 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): tablet f7f11b668f074d48af265b86c163d708 (table test-table [id=ad16cc4dc5f64f0f9e27038ee78c0e82]) successfully deleted
W20260812 21:11:54.069072 9293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:37865 (error 108)
I20260812 21:11:54.069494 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 9678
I20260812 21:11:54.069658 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:11:54.069489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:37865 (local address 127.0.0.1:60850)
0812 21:11:54.069561 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 21:11:54.069616 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 21:11:54.086526 9293 catalog_manager.cc:4584] TS 665d9eb8786f406f94a5b9245fc02e0d (127.11.207.129:37865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7f11b668f074d48af265b86c163d708: Network error: Client connection negotiation failed: client connection to 127.11.207.129:37865: connect: Connection refused (error 111)
I20260812 21:11:54.090929 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 9939
I20260812 21:11:54.120222 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 9258
I20260812 21:12:14.128145 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44453
--webserver_interface=127.11.207.190
--webserver_port=33633
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:44453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:14.213037 10250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:14.213248 10250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:14.213272 10250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:14.214465 10250 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 21:12:14.214516 10250 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 21:12:14.214527 10250 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 21:12:14.214538 10250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:14.214550 10250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:14.214573 10250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:14.214597 10250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:14.214619 10250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:14.216179 10250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:44453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=33633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:14.216435 10250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:14.216753 10250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:12:14.221709 10250 server_base.cc:1034] running on GCE node
W20260812 21:12:14.221661 10255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.221674 10256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.221692 10261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:14.222084 10250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:14.222393 10250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:14.223547 10250 hybrid_clock.cc:648] HybridClock initialized: now 1786569134223533 us; error 28 us; skew 500 ppm
Aug 12 21:12:14 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569134, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:14.226136 10250 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:12:14.226598 10250 webserver.cc:460] Webserver started at http://127.11.207.190:33633/ using document root <none> and password file <none>
I20260812 21:12:14.226833 10250 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:14.226886 10250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:14.228255 10250 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:12:14.228969 10279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.229147 10250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:12:14.229251 10250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "ae5366270f46483398ab62d4e51646df"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "26d5b65e5560bb2f5934746b0767857c"
server_key_iv: "d1a18ea639cc8e9603449338ee8ea4b5"
server_key_version: "encryptionkey@0"
I20260812 21:12:14.229591 10250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:14.239032 10250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:14.240883 10250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:14.241019 10250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:14.245059 10250 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:44453
I20260812 21:12:14.245092 10367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:44453 every 8 connection(s)
I20260812 21:12:14.245466 10250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:14.246913 10368 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:12:14.247596 10368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap starting.
I20260812 21:12:14.254287 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 10250
I20260812 21:12:14.254575 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:37865
--local_ip_for_outbound_sockets=127.11.207.129
--tserver_master_addrs=127.11.207.190:44453
--webserver_port=45607
--webserver_interface=127.11.207.129
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:12:14.257405 10368 log.cc:824] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:14.260115 10368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:12:14.260571 10368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Bootstrap complete.
I20260812 21:12:14.262946 10368 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:12:14.263235 10368 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5366270f46483398ab62d4e51646df, State: Initialized, Role: FOLLOWER
I20260812 21:12:14.263423 10368 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:12:14.263492 10368 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:14.263535 10368 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:14.263585 10368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:12:14.264675 10368 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:12:14.264775 10368 leader_election.cc:302] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae5366270f46483398ab62d4e51646df; no voters:
I20260812 21:12:14.264956 10368 leader_election.cc:288] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 21:12:14.265036 10375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 21:12:14.265192 10375 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [term 3 LEADER]: Becoming Leader. State: Replica: ae5366270f46483398ab62d4e51646df, State: Running, Role: LEADER
I20260812 21:12:14.265246 10368 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:14.265300 10375 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } }
I20260812 21:12:14.265566 10375 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae5366270f46483398ab62d4e51646df. Latest consensus state: current_term: 3 leader_uuid: "ae5366270f46483398ab62d4e51646df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } } }
I20260812 21:12:14.265614 10378 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ae5366270f46483398ab62d4e51646df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5366270f46483398ab62d4e51646df" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44453 } } }
I20260812 21:12:14.265707 10378 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:14.266055 10385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:14.265625 10375 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:14.266885 10385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:14.266987 10385 catalog_manager.cc:1259] Loaded cluster ID: 449d34439dd94aadaf7c267c00e48144
I20260812 21:12:14.267019 10385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:14.268244 10385 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:14.268466 10385 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Loaded TSK: 0
I20260812 21:12:14.268515 10385 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ae5366270f46483398ab62d4e51646df: Loaded TSK: 1
F20260812 21:12:14.268760 10385 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd9858f49ad google::LogMessage::Fail() at ??:0
@ 0x7fd9858f788a google::LogMessage::SendToLog() at ??:0
@ 0x7fd9858f44de google::LogMessage::Flush() at ??:0
@ 0x7fd9858f5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd988061104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd98805ed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd98d84e1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd98d84e48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd98d84e594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fd98d814c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd98d81ead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd9877c20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd9877b7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd98861b6db start_thread at ??:0
@ 0x7fd9843f261f clone at ??:0
W20260812 21:12:14.375612 10372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:14.375839 10372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:14.375866 10372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:14.377092 10372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:14.377151 10372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:14.377179 10372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:14.378945 10372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:37865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=45607
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:14.379197 10372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:14.379436 10372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:14.384403 10445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.384413 10437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.384413 10438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:14.384657 10372 server_base.cc:1034] running on GCE node
I20260812 21:12:14.384986 10372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:14.385241 10372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:14.386394 10372 hybrid_clock.cc:648] HybridClock initialized: now 1786569134386380 us; error 33 us; skew 500 ppm
Aug 12 21:12:14 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569134, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:14.389048 10372 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:12:14.389541 10372 webserver.cc:460] Webserver started at http://127.11.207.129:45607/ using document root <none> and password file <none>
I20260812 21:12:14.389757 10372 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:14.389808 10372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:14.391594 10372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:14.392422 10457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.392606 10372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:14.392695 10372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "665d9eb8786f406f94a5b9245fc02e0d"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "f7fa5318979bb05c7e7d9a64f99ea7ff"
server_key_iv: "6bf370eb9cfb765e34c2618fdccbcd06"
server_key_version: "encryptionkey@0"
I20260812 21:12:14.393061 10372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:14.404703 10372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:14.406852 10372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:14.406992 10372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:14.407214 10372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:14.407665 10467 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:12:14.409708 10372 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:12:14.409762 10372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:12:14.409797 10372 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:12:14.410490 10372 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:12:14.410523 10372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:12:14.410621 10467 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap starting.
I20260812 21:12:14.417882 10372 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:37865
I20260812 21:12:14.418035 10660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:37865 every 8 connection(s)
I20260812 21:12:14.418313 10372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:14.421042 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 10372
I20260812 21:12:14.421311 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:40559
--local_ip_for_outbound_sockets=127.11.207.130
--tserver_master_addrs=127.11.207.190:44453
--webserver_port=38075
--webserver_interface=127.11.207.130
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:12:14.422406 10467 log.cc:824] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:14.425385 10467 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:12:14.425704 10467 tablet_bootstrap.cc:492] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Bootstrap complete.
I20260812 21:12:14.427047 10467 ts_tablet_manager.cc:1397] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260812 21:12:14.428664 10467 raft_consensus.cc:348] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:14.429035 10467 raft_consensus.cc:729] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665d9eb8786f406f94a5b9245fc02e0d, State: Initialized, Role: FOLLOWER
I20260812 21:12:14.429315 10467 consensus_queue.cc:260] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:14.429636 10467 ts_tablet_manager.cc:1428] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
W20260812 21:12:14.500568 10668 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:12:14.418954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:47159)
0812 21:12:14.419122 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.419163 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:14.419618 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:14.500330 (+ 80712us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
W20260812 21:12:14.500864 10666 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:44453 (0 consecutive failures): Network error: Failed to ping master at 127.11.207.190:44453: Client connection negotiation failed: client connection to 127.11.207.190:44453: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:12:14.501219 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.501074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:49489)
0812 21:12:14.501128 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.501180 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:12:14.501566 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.501469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:59351)
0812 21:12:14.501512 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.501541 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 21:12:14.501710 10666 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:12:14.513150 10672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:14.513353 10672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:14.513376 10672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:14.514539 10672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:14.514582 10672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:14.514611 10672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:14.516086 10672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:40559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=38075
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:14.516310 10672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:14.516557 10672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:14.521507 10687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.521548 10683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:14.521576 10672 server_base.cc:1034] running on GCE node
W20260812 21:12:14.521507 10682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:14.521924 10672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:14.522163 10672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:14.523313 10672 hybrid_clock.cc:648] HybridClock initialized: now 1786569134523297 us; error 27 us; skew 500 ppm
Aug 12 21:12:14 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569134, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:14.525811 10672 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:12:14.526261 10672 webserver.cc:460] Webserver started at http://127.11.207.130:38075/ using document root <none> and password file <none>
I20260812 21:12:14.526461 10672 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:14.526511 10672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:14.527911 10672 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:14.528527 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.528705 10672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:14.528805 10672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "18828fc1652841d9bb58561a468f495e"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "e6f29ceebc352737b06e0152ee257498"
server_key_iv: "ac0e9cca3b8e10ce04d6b1935ad7d8f2"
server_key_version: "encryptionkey@0"
I20260812 21:12:14.529147 10672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:14.537911 10672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:14.539678 10672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:14.539824 10672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:14.540043 10672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:14.540334 10672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:14.540426 10672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.540457 10672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:14.540480 10672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.546300 10672 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:40559
I20260812 21:12:14.546365 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:40559 every 8 connection(s)
I20260812 21:12:14.546689 10672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:12:14.546932 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 10672
I20260812 21:12:14.547245 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:39273
--local_ip_for_outbound_sockets=127.11.207.131
--tserver_master_addrs=127.11.207.190:44453
--webserver_port=36035
--webserver_interface=127.11.207.131
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:12:14.547782 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.547395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:60717)
0812 21:12:14.547551 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.547630 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260812 21:12:14.548142 10914 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:44453 (0 consecutive failures): Network error: Failed to ping master at 127.11.207.190:44453: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
I20260812 21:12:14.548424 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.548314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:60343)
0812 21:12:14.548383 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.548405 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 21:12:14.548848 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.548774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:43031)
0812 21:12:14.548810 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.548831 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 21:12:14.548982 10914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:12:14.632897 10918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:14.633095 10918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:14.633117 10918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:14.634231 10918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:14.634266 10918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:14.634290 10918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:14.635696 10918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:39273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=36035
--tserver_master_addrs=127.11.207.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:14.635882 10918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:14.636070 10918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:14.641023 10933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.641012 10931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:14.641021 10936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:14.641242 10918 server_base.cc:1034] running on GCE node
I20260812 21:12:14.641446 10918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:14.641656 10918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:14.642824 10918 hybrid_clock.cc:648] HybridClock initialized: now 1786569134642805 us; error 32 us; skew 500 ppm
Aug 12 21:12:14 dist-test-slave-lkx2 krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569134, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:14.645380 10918 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:12:14.645828 10918 webserver.cc:460] Webserver started at http://127.11.207.131:36035/ using document root <none> and password file <none>
I20260812 21:12:14.646072 10918 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:14.646124 10918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:14.647486 10918 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:14.648164 10950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.648326 10918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:14.648445 10918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ec80641e788f441181390b8638bab15f"
format_stamp: "Formatted at 2026-08-12 21:11:32 on dist-test-slave-lkx2"
server_key: "36ada641079de0cdc34b804cd0e59d36"
server_key_iv: "00e6d6dbc74fc6dab0f997d510c39925"
server_key_version: "encryptionkey@0"
I20260812 21:12:14.648806 10918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:14.665563 10918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:14.668035 10918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:14.668191 10918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:14.668509 10918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:14.668865 10918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:14.668906 10918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.668939 10918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:14.668977 10918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:14.675071 10918 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:39273
I20260812 21:12:14.675136 11168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:39273 every 8 connection(s)
I20260812 21:12:14.675460 10918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:14.676542 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.676101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:48795)
0812 21:12:14.676264 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.676336 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
W20260812 21:12:14.676908 11169 heartbeater.cc:644] Failed to heartbeat to 127.11.207.190:44453 (0 consecutive failures): Network error: Failed to ping master at 127.11.207.190:44453: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
I20260812 21:12:14.677167 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.677084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:52431)
0812 21:12:14.677125 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.677152 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:12:14.677392 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:14.677329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:51385)
0812 21:12:14.677360 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:14.677380 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 21:12:14.677490 11169 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:12:14.683338 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 10918
I20260812 21:12:15.502365 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:15.502089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:37467)
0812 21:12:15.502250 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:15.502297 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260812 21:12:15.549615 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:15.549372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:54289)
0812 21:12:15.549509 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:15.549554 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260812 21:12:15.678068 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:15.677844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:51093)
0812 21:12:15.677977 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:15.678012 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260812 21:12:16.228679 11178 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:16.228826 11178 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:16.229120 11178 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559), ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
Aug 12 21:12:16 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569134, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.130@KRBTEST.COM
Aug 12 21:12:16 dist-test-slave-lkx2 krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569134, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.131@KRBTEST.COM
I20260812 21:12:16.243438 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.229444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:45803 (local address 127.11.207.130:40559)
0812 21:12:16.229596 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:16.229609 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:16.229632 (+ 23us) server_negotiation.cc:407] Connection header received
0812 21:12:16.230085 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:16.230087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:16.230126 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:16.230257 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:16.231001 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.231935 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.232964 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.233353 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.233471 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:16.239812 (+ 6341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:16.239819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:16.239819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:16.239833 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:16.241592 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:16.242077 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:16.242081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:16.242081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:16.242202 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:16.242489 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:16.242490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:16.242490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:16.242811 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:16.242873 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:16.243189 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 21:12:16.243314 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260812 21:12:16.243424 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.229280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:39273 (local address 127.11.207.129:46695)
0812 21:12:16.229413 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:16.229424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:16.229482 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:16.230332 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:16.230335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:16.230714 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:16.230880 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:16.230886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.232016 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.232017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:16.232883 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:16.232886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.233442 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.233443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:16.233552 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:16.238027 (+ 4475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:16.238033 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:16.240047 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:16.241981 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:16.241982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:16.241982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:16.242210 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:16.242489 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:16.242490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:16.242490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:16.242553 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:16.242947 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:16.242948 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:16.243026 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 21:12:16.243118 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 21:12:16.243325 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":6,"thread_start_us":67,"threads_started":1}
I20260812 21:12:16.243443 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.229410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:46695 (local address 127.11.207.131:39273)
0812 21:12:16.229617 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:16.229627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:16.229646 (+ 19us) server_negotiation.cc:407] Connection header received
0812 21:12:16.230080 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:16.230082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:16.230126 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:16.230266 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:16.231001 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.231927 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.232958 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.233352 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.233471 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:16.240159 (+ 6688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:16.240165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:16.240165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:16.240180 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:16.241893 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:16.242304 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:16.242307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:16.242307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:16.242419 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:16.242619 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:16.242619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:16.242620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:16.242867 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:16.242949 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:16.243188 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 21:12:16.243328 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":61,"threads_started":1}
I20260812 21:12:16.243515 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.229280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40559 (local address 127.11.207.129:45803)
0812 21:12:16.229447 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:16.229459 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:16.229532 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:16.230332 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:16.230335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:16.230680 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:16.230880 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:16.230886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.232030 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.232031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:16.232881 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:16.232884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:16.233446 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:16.233447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:16.233553 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:16.238016 (+ 4463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:16.238026 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:16.239673 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:16.241687 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:16.241695 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:16.241698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:16.241989 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:16.242283 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:16.242283 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:16.242284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:16.242422 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:16.242886 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:16.242894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:16.243029 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:12:16.243118 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:12:16.243401 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":199,"thread_start_us":78,"threads_started":1}
W20260812 21:12:16.244271 10459 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec80641e788f441181390b8638bab15f (127.11.207.131:39273): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
W20260812 21:12:16.244532 10460 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
I20260812 21:12:16.244593 10460 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d; no voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f
I20260812 21:12:16.244706 11178 raft_consensus.cc:2726] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:12:16.503067 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.502898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:47853)
0812 21:12:16.502975 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:16.503025 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 21:12:16.550437 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.550156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:60211)
0812 21:12:16.550322 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:16.550383 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260812 21:12:16.678747 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:16.678517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:58545)
0812 21:12:16.678636 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:16.678695 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260812 21:12:17.503865 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:17.503615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:34803)
0812 21:12:17.503769 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:17.503816 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260812 21:12:17.551252 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:17.550981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:55301)
0812 21:12:17.551140 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:17.551200 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260812 21:12:17.679508 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:17.679296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:58093)
0812 21:12:17.679422 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:17.679464 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260812 21:12:18.218106 11196 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:18.218235 11196 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:18.218401 11196 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559), ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
W20260812 21:12:18.218834 10460 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
W20260812 21:12:18.218900 10459 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec80641e788f441181390b8638bab15f (127.11.207.131:39273): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
I20260812 21:12:18.218959 10459 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d; no voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f
I20260812 21:12:18.219048 11196 raft_consensus.cc:2726] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:12:18.504609 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:18.504329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:59165)
0812 21:12:18.504485 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:18.504528 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260812 21:12:18.552131 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:18.551836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:44865)
0812 21:12:18.552011 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:18.552065 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260812 21:12:18.680229 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:18.679976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:60637)
0812 21:12:18.680098 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:18.680145 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260812 21:12:19.505357 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:19.505128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:40015)
0812 21:12:19.505264 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:19.505306 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260812 21:12:19.552860 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:19.552593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:59827)
0812 21:12:19.552757 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:19.552805 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":63,"threads_started":1}
I20260812 21:12:19.680999 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:19.680759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:46121)
0812 21:12:19.680911 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:19.680954 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260812 21:12:20.506073 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:20.505818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:59209)
0812 21:12:20.505975 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:20.506021 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260812 21:12:20.553558 11217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:20.553344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:36457)
0812 21:12:20.553476 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:20.553512 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260812 21:12:20.575620 11219 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:20.575721 11219 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:20.575896 11219 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559), ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
W20260812 21:12:20.576288 10460 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
W20260812 21:12:20.576387 10459 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec80641e788f441181390b8638bab15f (127.11.207.131:39273): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
I20260812 21:12:20.576459 10459 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d; no voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f
I20260812 21:12:20.576547 11219 raft_consensus.cc:2726] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:12:20.681713 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:20.681458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:60283)
0812 21:12:20.681604 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:20.681650 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260812 21:12:21.506811 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:21.506559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:37647)
0812 21:12:21.506696 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:21.506764 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260812 21:12:21.554307 11226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:21.554052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:54247)
0812 21:12:21.554197 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:21.554245 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260812 21:12:21.682379 11228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:21.682145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:37805)
0812 21:12:21.682285 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:21.682330 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260812 21:12:22.507543 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:22.507309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:33753)
0812 21:12:22.507451 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:22.507494 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260812 21:12:22.555097 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:22.554826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:49417)
0812 21:12:22.554996 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:22.555045 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260812 21:12:22.623869 11233 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:22.623991 11233 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:22.624172 11233 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559), ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
W20260812 21:12:22.624593 10459 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec80641e788f441181390b8638bab15f (127.11.207.131:39273): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
W20260812 21:12:22.624675 10460 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
I20260812 21:12:22.624720 10460 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d; no voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f
I20260812 21:12:22.624810 11233 raft_consensus.cc:2726] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:12:22.683087 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:22.682877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:50851)
0812 21:12:22.682996 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:22.683038 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260812 21:12:23.508244 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:23.508016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:33353)
0812 21:12:23.508151 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:23.508194 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260812 21:12:23.555886 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:23.555630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:51597)
0812 21:12:23.555781 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:23.555836 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260812 21:12:23.683699 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:23.683495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:44917)
0812 21:12:23.683616 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:23.683651 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260812 21:12:24.508960 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:24.508712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:48437)
0812 21:12:24.508865 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:24.508909 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260812 21:12:24.556706 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:24.556435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:54107)
0812 21:12:24.556600 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:24.556653 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260812 21:12:24.684422 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:24.684175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:59277)
0812 21:12:24.684316 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:24.684375 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260812 21:12:25.509639 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:25.509394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:57307)
0812 21:12:25.509545 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:25.509588 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260812 21:12:25.557390 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:25.557158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:50499)
0812 21:12:25.557295 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:25.557343 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260812 21:12:25.685077 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:25.684881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:48281)
0812 21:12:25.684995 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:25.685027 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260812 21:12:26.402465 11253 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:26.402566 11253 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:26.402734 11253 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559), ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
W20260812 21:12:26.403175 10460 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
W20260812 21:12:26.403271 10459 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec80641e788f441181390b8638bab15f (127.11.207.131:39273): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
I20260812 21:12:26.403319 10459 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d; no voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f
I20260812 21:12:26.403395 11253 raft_consensus.cc:2726] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:12:26.510413 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:26.510186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:56945)
0812 21:12:26.510321 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:26.510363 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260812 21:12:26.558130 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:26.557910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:45563)
0812 21:12:26.558034 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:26.558081 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260812 21:12:26.685648 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:26.685461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:59059)
0812 21:12:26.685570 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:26.685602 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260812 21:12:27.511089 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:27.510860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:53835)
0812 21:12:27.510997 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:27.511038 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260812 21:12:27.558867 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:27.558631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:47689)
0812 21:12:27.558767 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:27.558817 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260812 21:12:27.686398 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:27.686139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:53011)
0812 21:12:27.686299 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:27.686349 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260812 21:12:28.511817 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:28.511594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:46445)
0812 21:12:28.511727 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:28.511767 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260812 21:12:28.559572 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:28.559357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:45471)
0812 21:12:28.559480 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:28.559523 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260812 21:12:28.687211 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:28.686921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:54449)
0812 21:12:28.687101 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:28.687149 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260812 21:12:29.223335 11267 raft_consensus.cc:482] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:29.223459 11267 raft_consensus.cc:504] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18828fc1652841d9bb58561a468f495e" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40559 } } peers { permanent_uuid: "ec80641e788f441181390b8638bab15f" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39273 } } peers { permanent_uuid: "665d9eb8786f406f94a5b9245fc02e0d" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 37865 } }
I20260812 21:12:29.223646 11267 leader_election.cc:288] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559), ec80641e788f441181390b8638bab15f (127.11.207.131:39273)
W20260812 21:12:29.224085 10459 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec80641e788f441181390b8638bab15f (127.11.207.131:39273): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
W20260812 21:12:29.224213 10460 leader_election.cc:341] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18828fc1652841d9bb58561a468f495e (127.11.207.130:40559): Not found: Tablet not found: f7f11b668f074d48af265b86c163d708
I20260812 21:12:29.224287 10460 leader_election.cc:302] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 665d9eb8786f406f94a5b9245fc02e0d; no voters: 18828fc1652841d9bb58561a468f495e, ec80641e788f441181390b8638bab15f
I20260812 21:12:29.224387 11267 raft_consensus.cc:2726] T f7f11b668f074d48af265b86c163d708 P 665d9eb8786f406f94a5b9245fc02e0d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:12:29.512593 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:29.512386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.129:54835)
0812 21:12:29.512502 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:29.512541 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260812 21:12:29.560436 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:29.560143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.130:34927)
0812 21:12:29.560303 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:29.560382 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260812 21:12:29.687963 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:29.687733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:44453 (local address 127.11.207.131:60577)
0812 21:12:29.687866 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:29.687905 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.190:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260812 21:12:29.689123 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 10372
I20260812 21:12:29.711205 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 10672
I20260812 21:12:29.723214 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 10918
2026-08-12T21:12:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59290 ms)
[----------] 30 tests from SecurityITest (244928 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:29 dist-test-slave-lkx2 krb5kdc[11278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:29 dist-test-slave-lkx2 krb5kdc[11278](info): set up 2 sockets
Aug 12 21:12:29 dist-test-slave-lkx2 krb5kdc[11278](info): commencing operation
krb5kdc: starting...
W20260812 21:12:30.946089 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:12:31.070749 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:46243
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:31.169910 11309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:31.170126 11309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:31.170147 11309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:31.171298 11309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:31.171341 11309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:31.171355 11309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:31.171365 11309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:31.171376 11309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:31.172924 11309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:46243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:31.173180 11309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:31.173408 11309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:31.178512 11324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.178529 11319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.178530 11322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:31.178580 11309 server_base.cc:1034] running on GCE node
I20260812 21:12:31.178977 11309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:31.179288 11309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:31.180395 11309 hybrid_clock.cc:648] HybridClock initialized: now 1786569151180366 us; error 45 us; skew 500 ppm
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:31.182757 11309 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:12:31.183197 11309 webserver.cc:460] Webserver started at http://127.11.207.190:39543/ using document root <none> and password file <none>
I20260812 21:12:31.183427 11309 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:31.183476 11309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:31.183604 11309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:31.184710 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "04f4873f0082415eaa616d7094de7a0e"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "48dd8250a84e308a0eb5185893449310"
server_key_iv: "5e1fc9d22549caf02aa5a9b15b11d6dc"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.185047 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "04f4873f0082415eaa616d7094de7a0e"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "48dd8250a84e308a0eb5185893449310"
server_key_iv: "5e1fc9d22549caf02aa5a9b15b11d6dc"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.186395 11309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:12:31.187120 11343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.187279 11309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:31.187364 11309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "04f4873f0082415eaa616d7094de7a0e"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "48dd8250a84e308a0eb5185893449310"
server_key_iv: "5e1fc9d22549caf02aa5a9b15b11d6dc"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.187424 11309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:31.212996 11309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:31.214905 11309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:31.215049 11309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:31.219084 11309 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:46243
I20260812 21:12:31.219123 11437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:46243 every 8 connection(s)
I20260812 21:12:31.219476 11309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:31.220402 11438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:31.222723 11438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e: Bootstrap starting.
I20260812 21:12:31.223297 11438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:31.223567 11438 log.cc:824] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:31.224318 11438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e: No bootstrap required, opened a new log
I20260812 21:12:31.225996 11438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4873f0082415eaa616d7094de7a0e" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 46243 } }
I20260812 21:12:31.226140 11438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:31.226162 11438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f4873f0082415eaa616d7094de7a0e, State: Initialized, Role: FOLLOWER
I20260812 21:12:31.226306 11438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4873f0082415eaa616d7094de7a0e" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 46243 } }
I20260812 21:12:31.226373 11438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:31.226395 11438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:31.226444 11438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:31.227008 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 11309
I20260812 21:12:31.227033 11438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4873f0082415eaa616d7094de7a0e" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 46243 } }
I20260812 21:12:31.227121 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:31.227135 11438 leader_election.cc:302] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04f4873f0082415eaa616d7094de7a0e; no voters:
I20260812 21:12:31.227325 11438 leader_election.cc:288] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:31.227355 12094 external_mini_cluster.cc:1428] Setting key 62f7a87a82641aa0249f3272b96eb93a
I20260812 21:12:31.227361 11445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:31.227558 11445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [term 1 LEADER]: Becoming Leader. State: Replica: 04f4873f0082415eaa616d7094de7a0e, State: Running, Role: LEADER
I20260812 21:12:31.227589 11438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:31.227675 11445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4873f0082415eaa616d7094de7a0e" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 46243 } }
I20260812 21:12:31.228296 11446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04f4873f0082415eaa616d7094de7a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4873f0082415eaa616d7094de7a0e" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 46243 } } }
I20260812 21:12:31.228413 11446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:31.228597 11449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f4873f0082415eaa616d7094de7a0e. Latest consensus state: current_term: 1 leader_uuid: "04f4873f0082415eaa616d7094de7a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f4873f0082415eaa616d7094de7a0e" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 46243 } } }
I20260812 21:12:31.228658 11449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:31.228935 11469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:31.229537 11469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:31.232874 11469 catalog_manager.cc:1347] Generated new cluster ID: 5c861e8f4e184672871f252a2f134ee4
I20260812 21:12:31.232966 11469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:31.236928 11469 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:31.237607 11469 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:31.238940 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.227909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.11.207.190:46243)
0812 21:12:31.228099 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.228109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.228146 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:12:31.228275 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.228283 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.228372 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.228561 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:31.229383 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.230783 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.231592 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.232196 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.232334 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.235371 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:31.235375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:31.235375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:31.235390 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:31.237399 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.237763 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.237767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.237767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.237878 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.238074 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.238075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.238075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.238332 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:31.238497 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.238599 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.238770 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":63,"threads_started":1}
I20260812 21:12:31.253083 11469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e: Generated new TSK 0
I20260812 21:12:31.253392 11469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:12:31.287279 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:46243
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:31.373853 11492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:31.374068 11492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:31.374100 11492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:31.375277 11492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:31.375319 11492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:31.375355 11492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:31.376874 11492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:31.377105 11492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:31.377321 11492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:31.382364 11500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.382377 11507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.382377 11503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:31.382436 11492 server_base.cc:1034] running on GCE node
I20260812 21:12:31.382833 11492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:31.383105 11492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:31.384251 11492 hybrid_clock.cc:648] HybridClock initialized: now 1786569151384239 us; error 29 us; skew 500 ppm
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:31.386904 11492 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:12:31.387385 11492 webserver.cc:460] Webserver started at http://127.11.207.129:35753/ using document root <none> and password file <none>
I20260812 21:12:31.387588 11492 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:31.387637 11492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:31.387771 11492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:31.388902 11492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "72ceb844eace4b9c91022f6e3f96d5bc"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "ec5ee6a90cbb933c9ff63512f6c06997"
server_key_iv: "dc9f2d62e3ee0e82d1b0172b71891ee6"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.389245 11492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "72ceb844eace4b9c91022f6e3f96d5bc"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "ec5ee6a90cbb933c9ff63512f6c06997"
server_key_iv: "dc9f2d62e3ee0e82d1b0172b71891ee6"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.390550 11492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:12:31.391341 11519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.391521 11492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:31.391614 11492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "72ceb844eace4b9c91022f6e3f96d5bc"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "ec5ee6a90cbb933c9ff63512f6c06997"
server_key_iv: "dc9f2d62e3ee0e82d1b0172b71891ee6"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.391705 11492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:31.403076 11492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:31.405272 11492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:31.405428 11492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:31.405670 11492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:31.406102 11492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:31.406142 11492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.406188 11492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:31.406213 11492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.412042 11492 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:41427
I20260812 21:12:31.412091 11719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:41427 every 8 connection(s)
I20260812 21:12:31.412446 11492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:31.413568 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 11492
I20260812 21:12:31.413674 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:12:31.413861 12094 external_mini_cluster.cc:1428] Setting key c674cc832691b916b5dc1f38dcea43bd
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:31.422115 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.413272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:43917 (local address 127.11.207.190:46243)
0812 21:12:31.413329 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.413330 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.413501 (+ 171us) server_negotiation.cc:407] Connection header received
0812 21:12:31.414098 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.414100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.414114 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.414168 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:31.415256 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.415921 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.417002 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.417197 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.417261 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.419427 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:31.419428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:31.419428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:31.419437 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:31.420839 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.421206 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.421207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.421207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.421259 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.421473 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.421474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.421474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.421588 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:31.421646 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.421938 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.422025 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
I20260812 21:12:31.422282 11729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.413170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:46243 (local address 127.11.207.129:43917)
0812 21:12:31.413361 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.413416 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.413970 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.414251 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.414256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.414957 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:31.415157 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.415162 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.416019 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.416020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.416912 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.416915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.417278 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.417279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.417410 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.417903 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:31.417911 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:31.419338 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:31.420913 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:31.420921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:31.420923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:31.421134 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:31.421313 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:31.421314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:31.421315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:31.421418 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:31.421641 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:31.421649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:31.421812 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.421905 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.422061 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260812 21:12:31.422792 11720 heartbeater.cc:347] Connected to a master server at 127.11.207.190:46243
I20260812 21:12:31.422892 11720 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:31.423070 11720 heartbeater.cc:511] Master 127.11.207.190:46243 requested a full tablet report, sending...
I20260812 21:12:31.423662 11378 ts_manager.cc:194] Registered new tserver with Master: 72ceb844eace4b9c91022f6e3f96d5bc (127.11.207.129:41427)
I20260812 21:12:31.424259 11378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:43917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:12:31.443851 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:46243
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:31.527943 11737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:31.528177 11737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:31.528205 11737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:31.529516 11737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:31.529565 11737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:31.529592 11737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:31.531106 11737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:31.531344 11737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:31.531563 11737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:31.536693 11746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.536782 11747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.536814 11750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:31.536860 11737 server_base.cc:1034] running on GCE node
I20260812 21:12:31.537211 11737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:31.537472 11737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:31.538637 11737 hybrid_clock.cc:648] HybridClock initialized: now 1786569151538623 us; error 29 us; skew 500 ppm
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:31.541266 11737 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:12:31.541723 11737 webserver.cc:460] Webserver started at http://127.11.207.130:42681/ using document root <none> and password file <none>
I20260812 21:12:31.541937 11737 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:31.541985 11737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:31.542115 11737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:31.543192 11737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "b404e9321e00451eac56a84d9ce58c55"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "9cd67509a933144d0045e6f379057708"
server_key_iv: "a7707b0660966a8beacdbc600fe9c25b"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.543524 11737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "b404e9321e00451eac56a84d9ce58c55"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "9cd67509a933144d0045e6f379057708"
server_key_iv: "a7707b0660966a8beacdbc600fe9c25b"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.544878 11737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:31.545575 11758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.545751 11737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:31.545840 11737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "b404e9321e00451eac56a84d9ce58c55"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "9cd67509a933144d0045e6f379057708"
server_key_iv: "a7707b0660966a8beacdbc600fe9c25b"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.545902 11737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:31.562753 11737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:31.564977 11737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:31.565120 11737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:31.565343 11737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:31.565647 11737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:31.565697 11737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.565727 11737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:31.565748 11737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.571422 11737 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:44103
I20260812 21:12:31.571483 11956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:44103 every 8 connection(s)
I20260812 21:12:31.571837 11737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:31.579814 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 11737
I20260812 21:12:31.579949 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:12:31.580194 12094 external_mini_cluster.cc:1428] Setting key b6fc5f2383193e672a6fccd9532f5d22
I20260812 21:12:31.581597 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.572588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:41353 (local address 127.11.207.190:46243)
0812 21:12:31.572633 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.572635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.572861 (+ 226us) server_negotiation.cc:407] Connection header received
0812 21:12:31.573354 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.573355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.573366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.573424 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:31.574373 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.574931 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.575914 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.576107 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.576166 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.578583 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:31.578585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:31.578586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:31.578593 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:31.580116 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.580605 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.580606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.580607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.580680 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.580965 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.580966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.580967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.581103 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:31.581162 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.581466 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.581533 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:12:31.581840 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.572575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:46243 (local address 127.11.207.130:41353)
0812 21:12:31.572745 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.572782 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.573267 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.573489 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.573492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.574063 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:31.574270 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.574275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.575031 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.575031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.575834 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.575837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.576210 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.576212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.576369 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.576883 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:31.576891 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:31.578496 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:31.580191 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:31.580199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:31.580201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:31.580510 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:31.580765 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:31.580765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:31.580766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:31.580902 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:31.581162 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:31.581171 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:31.581334 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.581442 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.581600 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260812 21:12:31.582479 11957 heartbeater.cc:347] Connected to a master server at 127.11.207.190:46243
I20260812 21:12:31.582583 11957 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:31.582792 11957 heartbeater.cc:511] Master 127.11.207.190:46243 requested a full tablet report, sending...
I20260812 21:12:31.583187 11378 ts_manager.cc:194] Registered new tserver with Master: b404e9321e00451eac56a84d9ce58c55 (127.11.207.130:44103)
I20260812 21:12:31.583665 11378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:41353
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:12:31.606739 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:46243
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:43733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:31.691411 11963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:31.691614 11963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:31.691637 11963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:31.692787 11963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:31.692827 11963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:31.692852 11963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:31.694303 11963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:43733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:46243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:31.694517 11963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:31.694772 11963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:31.699676 11971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:31.699661 11975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:31.699738 11963 server_base.cc:1034] running on GCE node
W20260812 21:12:31.699661 11973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:31.700065 11963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:31.700325 11963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:31.701534 11963 hybrid_clock.cc:648] HybridClock initialized: now 1786569151701517 us; error 35 us; skew 500 ppm
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:31.704058 11963 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:12:31.704521 11963 webserver.cc:460] Webserver started at http://127.11.207.131:43711/ using document root <none> and password file <none>
I20260812 21:12:31.704726 11963 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:31.704773 11963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:31.704901 11963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:31.705912 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "6022d83eb09042c191b6f3a5b91e8e46"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "046fe9f24a544a04f1fbebb282d0d681"
server_key_iv: "3aaadcd22e77111ccb355d263419a729"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.706256 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "6022d83eb09042c191b6f3a5b91e8e46"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "046fe9f24a544a04f1fbebb282d0d681"
server_key_iv: "3aaadcd22e77111ccb355d263419a729"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.707548 11963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:12:31.708310 11985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.708500 11963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:31.708588 11963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "6022d83eb09042c191b6f3a5b91e8e46"
format_stamp: "Formatted at 2026-08-12 21:12:31 on dist-test-slave-lkx2"
server_key: "046fe9f24a544a04f1fbebb282d0d681"
server_key_iv: "3aaadcd22e77111ccb355d263419a729"
server_key_version: "encryptionkey@0"
I20260812 21:12:31.708655 11963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:31.718075 11963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:31.720103 11963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:31.720243 11963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:31.720484 11963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:31.720794 11963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:31.720829 11963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.720856 11963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:31.720880 11963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:31.726516 11963 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:45707
I20260812 21:12:31.726575 12233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:45707 every 8 connection(s)
I20260812 21:12:31.726876 11963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:31.732569 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 11963
I20260812 21:12:31.732686 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:12:31.732931 12094 external_mini_cluster.cc:1428] Setting key 2e45c3d8607e602edbd1c198a8fafcab
Aug 12 21:12:31 dist-test-slave-lkx2 krb5kdc[11278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:31.736100 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.727568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:53919 (local address 127.11.207.190:46243)
0812 21:12:31.727623 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.727625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.727893 (+ 268us) server_negotiation.cc:407] Connection header received
0812 21:12:31.728471 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.728473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.728486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.728539 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:31.729607 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.730111 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.731085 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.731295 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.731352 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.733642 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:31.733644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:31.733645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:31.733653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:31.735025 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.735390 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.735391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.735391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.735439 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.735613 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.735614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.735614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.735713 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:31.735756 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.735976 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.736036 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:12:31.736294 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.727531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:46243 (local address 127.11.207.131:53919)
0812 21:12:31.727718 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.727796 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.728378 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.728612 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.728617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.729295 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:31.729522 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.729528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.730194 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.730195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.731003 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.731005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.731366 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.731367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.731497 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.731974 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:31.731982 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:31.733553 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:31.735095 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:31.735102 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:31.735103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:31.735334 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:31.735481 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:31.735481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:31.735482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:31.735573 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:31.735757 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:31.735762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:31.735879 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.735960 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.736079 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260812 21:12:31.736759 12234 heartbeater.cc:347] Connected to a master server at 127.11.207.190:46243
I20260812 21:12:31.736856 12234 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:31.737030 12234 heartbeater.cc:511] Master 127.11.207.190:46243 requested a full tablet report, sending...
I20260812 21:12:31.737362 11378 ts_manager.cc:194] Registered new tserver with Master: 6022d83eb09042c191b6f3a5b91e8e46 (127.11.207.131:45707)
I20260812 21:12:31.737721 11378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:53919
I20260812 21:12:31.745112 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:12:31.752169 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.746442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.11.207.190:46243)
0812 21:12:31.746495 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.746496 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.746527 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:12:31.746584 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.746584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.746593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.746637 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:31.747167 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.747648 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.748402 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.748639 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.748736 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.749919 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:31.749921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:31.749921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:31.749929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:31.751203 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.751526 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.751527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.751528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.751582 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:31.751716 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:31.751717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:31.751717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:31.751857 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:31.751930 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.752043 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.752103 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:12:31.753728 11378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:12:31.755321 11378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:12:31.761802 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.757909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44103 (local address 127.0.0.1:53626)
0812 21:12:31.758189 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.758198 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.758235 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.758731 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.758733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.758735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:31.758772 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.758774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.759939 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.759940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.760844 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.760846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.761498 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.761499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.761580 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.761594 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.761658 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.761714 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260812 21:12:31.761812 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.758052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.11.207.130:44103)
0812 21:12:31.758575 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.758580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.758589 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:12:31.758626 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.758628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.758643 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.758718 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:31.758843 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.759848 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.760917 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.761418 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.761486 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.761546 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.761681 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.761725 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":87,"threads_started":1}
I20260812 21:12:31.762179 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.757642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:45707 (local address 127.0.0.1:43792)
0812 21:12:31.757780 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.757827 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.757894 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.758428 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.758429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.758431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:31.758493 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.758495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.759493 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.759494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.760879 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.760880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.761905 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.761907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.761982 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.761986 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.762047 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.762101 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":8,"thread_start_us":40,"threads_started":1}
I20260812 21:12:31.762209 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.758042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.11.207.131:45707)
0812 21:12:31.758275 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.758280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.758292 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:12:31.758332 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.758334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.758349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.758429 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:31.758587 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.759395 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.760977 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.761728 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.761808 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.761863 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.762079 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.762130 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260812 21:12:31.762573 11832 tablet_service.cc:1467] Processing CreateTablet for tablet e4c0dbddcb7140fbb33a2e24efd3f178 (DEFAULT_TABLE table=test-table [id=c9916e36a2a54ab18c5fe664f3367529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:31.762853 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.757695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:41427 (local address 127.0.0.1:48580)
0812 21:12:31.758036 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.758046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.758100 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.758371 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.758376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.758378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:31.758451 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.758454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.759756 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.759757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.760862 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.760865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.762538 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.762540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.762603 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.762608 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.762660 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.762705 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":91,"threads_started":1}
I20260812 21:12:31.763008 11832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c0dbddcb7140fbb33a2e24efd3f178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:31.762965 12093 tablet_service.cc:1467] Processing CreateTablet for tablet e4c0dbddcb7140fbb33a2e24efd3f178 (DEFAULT_TABLE table=test-table [id=c9916e36a2a54ab18c5fe664f3367529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:31.763299 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.757822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.11.207.129:41427)
0812 21:12:31.758085 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.758090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.758103 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:12:31.758181 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.758184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.758202 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.758293 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:31.758535 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.759652 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.760977 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.761728 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.761808 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.763105 (+ 1297us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.763152 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.763207 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
I20260812 21:12:31.763355 12093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c0dbddcb7140fbb33a2e24efd3f178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:31.763978 11605 tablet_service.cc:1467] Processing CreateTablet for tablet e4c0dbddcb7140fbb33a2e24efd3f178 (DEFAULT_TABLE table=test-table [id=c9916e36a2a54ab18c5fe664f3367529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:31.764391 11605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c0dbddcb7140fbb33a2e24efd3f178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:31.766158 12266 tablet_bootstrap.cc:492] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Bootstrap starting.
I20260812 21:12:31.766533 12267 tablet_bootstrap.cc:492] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Bootstrap starting.
I20260812 21:12:31.767036 12266 tablet_bootstrap.cc:654] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:31.767251 12267 tablet_bootstrap.cc:654] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:31.767494 12274 tablet_bootstrap.cc:492] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc: Bootstrap starting.
I20260812 21:12:31.767890 12267 log.cc:824] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:31.767984 12266 log.cc:824] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:31.768225 12274 tablet_bootstrap.cc:654] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:31.768790 12274 log.cc:824] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:31.768803 12266 tablet_bootstrap.cc:492] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: No bootstrap required, opened a new log
I20260812 21:12:31.768805 12267 tablet_bootstrap.cc:492] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: No bootstrap required, opened a new log
I20260812 21:12:31.768868 12266 ts_tablet_manager.cc:1397] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:12:31.768868 12267 ts_tablet_manager.cc:1397] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:12:31.769415 12274 tablet_bootstrap.cc:492] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc: No bootstrap required, opened a new log
I20260812 21:12:31.769476 12274 ts_tablet_manager.cc:1397] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:12:31.770792 12274 raft_consensus.cc:348] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.770936 12274 raft_consensus.cc:374] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:31.770970 12274 raft_consensus.cc:729] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ceb844eace4b9c91022f6e3f96d5bc, State: Initialized, Role: FOLLOWER
I20260812 21:12:31.770896 12267 raft_consensus.cc:348] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.771081 12267 raft_consensus.cc:374] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:31.770896 12266 raft_consensus.cc:348] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.771121 12267 raft_consensus.cc:729] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b404e9321e00451eac56a84d9ce58c55, State: Initialized, Role: FOLLOWER
I20260812 21:12:31.771088 12274 consensus_queue.cc:260] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.771234 12267 consensus_queue.cc:260] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.771365 12274 ts_tablet_manager.cc:1428] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:12:31.771150 12266 raft_consensus.cc:374] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:31.771485 11720 heartbeater.cc:503] Master 127.11.207.190:46243 was elected leader, sending a full tablet report...
I20260812 21:12:31.771503 12266 raft_consensus.cc:729] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6022d83eb09042c191b6f3a5b91e8e46, State: Initialized, Role: FOLLOWER
I20260812 21:12:31.771543 12267 ts_tablet_manager.cc:1428] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:12:31.771654 12266 consensus_queue.cc:260] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.771714 11957 heartbeater.cc:503] Master 127.11.207.190:46243 was elected leader, sending a full tablet report...
I20260812 21:12:31.771970 12234 heartbeater.cc:503] Master 127.11.207.190:46243 was elected leader, sending a full tablet report...
I20260812 21:12:31.771952 12266 ts_tablet_manager.cc:1428] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260812 21:12:31.822394 11958 tablet.cc:2369] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:12:31.836670 12283 raft_consensus.cc:482] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:31.836831 12283 raft_consensus.cc:504] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.837265 12283 leader_election.cc:288] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6022d83eb09042c191b6f3a5b91e8e46 (127.11.207.131:45707), b404e9321e00451eac56a84d9ce58c55 (127.11.207.130:44103)
I20260812 21:12:31.840068 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.837602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:35065 (local address 127.11.207.131:45707)
0812 21:12:31.837645 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.837646 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.837654 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:12:31.837683 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.837683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.837690 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.837769 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:31.837937 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.838424 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.839341 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.839737 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.839809 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.839818 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.839965 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.840006 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:12:31.840068 11729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.837426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:45707 (local address 127.11.207.129:35065)
0812 21:12:31.837487 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.837501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.837555 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.837769 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.837770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.837773 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:31.837854 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.837856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.838503 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.838504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.839250 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.839252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.839822 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.839823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.839901 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.839904 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.839949 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.839990 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:12:31.840220 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.837512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:52513 (local address 127.11.207.130:44103)
0812 21:12:31.837581 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.837584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.837917 (+ 333us) server_negotiation.cc:407] Connection header received
0812 21:12:31.837949 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.837951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.837960 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.838006 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:31.838147 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.838615 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.839510 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.839898 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.839959 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.839968 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.840101 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.840141 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 21:12:31.840507 12136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c0dbddcb7140fbb33a2e24efd3f178" candidate_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022d83eb09042c191b6f3a5b91e8e46" is_pre_election: true
I20260812 21:12:31.840643 12136 raft_consensus.cc:2454] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ceb844eace4b9c91022f6e3f96d5bc in term 0.
I20260812 21:12:31.840776 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.837438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:44103 (local address 127.11.207.129:52513)
0812 21:12:31.837843 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.837852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:31.837899 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:31.838020 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:31.838021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:31.838022 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:31.838088 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.838090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.838690 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.838691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.839418 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:31.839420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.839976 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.839978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:31.840037 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.840040 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:31.840086 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:12:31.840253 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":131,"threads_started":1}
I20260812 21:12:31.840881 11521 leader_election.cc:302] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6022d83eb09042c191b6f3a5b91e8e46, 72ceb844eace4b9c91022f6e3f96d5bc; no voters:
I20260812 21:12:31.841020 12283 raft_consensus.cc:2781] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:12:31.841110 12283 raft_consensus.cc:482] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:12:31.841137 12283 raft_consensus.cc:3037] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:31.841171 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c0dbddcb7140fbb33a2e24efd3f178" candidate_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b404e9321e00451eac56a84d9ce58c55" is_pre_election: true
I20260812 21:12:31.841322 11873 raft_consensus.cc:2454] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ceb844eace4b9c91022f6e3f96d5bc in term 0.
I20260812 21:12:31.841997 12283 raft_consensus.cc:504] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.842137 12283 leader_election.cc:288] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [CANDIDATE]: Term 1 election: Requested vote from peers 6022d83eb09042c191b6f3a5b91e8e46 (127.11.207.131:45707), b404e9321e00451eac56a84d9ce58c55 (127.11.207.130:44103)
I20260812 21:12:31.842304 12136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c0dbddcb7140fbb33a2e24efd3f178" candidate_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022d83eb09042c191b6f3a5b91e8e46"
I20260812 21:12:31.842304 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c0dbddcb7140fbb33a2e24efd3f178" candidate_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b404e9321e00451eac56a84d9ce58c55"
I20260812 21:12:31.842392 11873 raft_consensus.cc:3037] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:31.842392 12136 raft_consensus.cc:3037] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:31.843317 12136 raft_consensus.cc:2454] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ceb844eace4b9c91022f6e3f96d5bc in term 1.
I20260812 21:12:31.843317 11873 raft_consensus.cc:2454] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ceb844eace4b9c91022f6e3f96d5bc in term 1.
I20260812 21:12:31.843513 11521 leader_election.cc:302] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6022d83eb09042c191b6f3a5b91e8e46, 72ceb844eace4b9c91022f6e3f96d5bc; no voters:
I20260812 21:12:31.843622 12283 raft_consensus.cc:2781] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:31.843787 12283 raft_consensus.cc:686] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [term 1 LEADER]: Becoming Leader. State: Replica: 72ceb844eace4b9c91022f6e3f96d5bc, State: Running, Role: LEADER
I20260812 21:12:31.843909 12283 consensus_queue.cc:237] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } }
I20260812 21:12:31.844774 11376 catalog_manager.cc:5495] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 72ceb844eace4b9c91022f6e3f96d5bc (127.11.207.129). New cstate: current_term: 1 leader_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ceb844eace4b9c91022f6e3f96d5bc" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 41427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 } health_report { overall_health: UNKNOWN } } }
I20260812 21:12:31.856323 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.853952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.11.207.129:41427)
0812 21:12:31.854001 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:31.854002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:31.854011 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:12:31.854051 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:31.854052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:31.854071 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:31.854142 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:12:31.854359 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.854904 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.855654 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:31.855888 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:31.855947 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:31.856064 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:12:31.856169 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:12:31.856230 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:31.856243 (+ 13us) server_negotiation.cc:299] Negotiation successful
0812 21:12:31.856259 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:12:31.858412 12136 raft_consensus.cc:1261] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 1 FOLLOWER]: Refusing update from remote peer 72ceb844eace4b9c91022f6e3f96d5bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:31.858412 11873 raft_consensus.cc:1261] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 1 FOLLOWER]: Refusing update from remote peer 72ceb844eace4b9c91022f6e3f96d5bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:31.858783 12283 consensus_queue.cc:1035] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6022d83eb09042c191b6f3a5b91e8e46" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 45707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:31.858934 12287 consensus_queue.cc:1035] T e4c0dbddcb7140fbb33a2e24efd3f178 P 72ceb844eace4b9c91022f6e3f96d5bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b404e9321e00451eac56a84d9ce58c55" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:31.862389 12293 mvcc.cc:205] Tried to move back new op lower bound from 7317787246009004032 to 7317787245953236992. Current Snapshot: MvccSnapshot[applied={T|T < 7317787246009004032}]
I20260812 21:12:31.864131 11376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:31.864243 11376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:31.865072 11376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04f4873f0082415eaa616d7094de7a0e: Sending DeleteTablet for 3 replicas of tablet e4c0dbddcb7140fbb33a2e24efd3f178
I20260812 21:12:31.865404 11605 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c0dbddcb7140fbb33a2e24efd3f178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:31 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:48580
I20260812 21:12:31.865435 12093 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c0dbddcb7140fbb33a2e24efd3f178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:31 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:43792
I20260812 21:12:31.865460 11832 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c0dbddcb7140fbb33a2e24efd3f178 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:31 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:53626
I20260812 21:12:31.865676 12309 tablet_replica.cc:330] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: stopping tablet replica
I20260812 21:12:31.865777 12309 raft_consensus.cc:2229] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:31.865875 12309 raft_consensus.cc:2258] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:31.866027 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 11492
I20260812 21:12:31.877240 12309 ts_tablet_manager.cc:1905] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:31.878976 12311 tablet_replica.cc:330] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: stopping tablet replica
I20260812 21:12:31.879176 12311 raft_consensus.cc:2229] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:31.879357 12311 raft_consensus.cc:2258] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:31.879585 12309 ts_tablet_manager.cc:1918] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:31.879688 12309 log.cc:1196] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/e4c0dbddcb7140fbb33a2e24efd3f178
I20260812 21:12:31.880089 12309 ts_tablet_manager.cc:1939] T e4c0dbddcb7140fbb33a2e24efd3f178 P 6022d83eb09042c191b6f3a5b91e8e46: Deleting consensus metadata
I20260812 21:12:31.890908 11345 catalog_manager.cc:4841] TS 6022d83eb09042c191b6f3a5b91e8e46 (127.11.207.131:45707): tablet e4c0dbddcb7140fbb33a2e24efd3f178 (table test-table [id=c9916e36a2a54ab18c5fe664f3367529]) successfully deleted
I20260812 21:12:31.891618 12311 ts_tablet_manager.cc:1905] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:31.893582 12311 ts_tablet_manager.cc:1918] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:31.893687 12311 log.cc:1196] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/e4c0dbddcb7140fbb33a2e24efd3f178
I20260812 21:12:31.893983 12311 ts_tablet_manager.cc:1939] T e4c0dbddcb7140fbb33a2e24efd3f178 P b404e9321e00451eac56a84d9ce58c55: Deleting consensus metadata
I20260812 21:12:31.894533 11346 catalog_manager.cc:4841] TS b404e9321e00451eac56a84d9ce58c55 (127.11.207.130:44103): tablet e4c0dbddcb7140fbb33a2e24efd3f178 (table test-table [id=c9916e36a2a54ab18c5fe664f3367529]) successfully deleted
W20260812 21:12:31.896718 11346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:41427 (error 108)
I20260812 21:12:31.896849 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 11737
I20260812 21:12:31.905821 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:31.905659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:41427 (local address 127.0.0.1:48600)
0812 21:12:31.905721 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:31.905777 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:41427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 21:12:31.905962 11346 catalog_manager.cc:4584] TS 72ceb844eace4b9c91022f6e3f96d5bc (127.11.207.129:41427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4c0dbddcb7140fbb33a2e24efd3f178: Network error: Client connection negotiation failed: client connection to 127.11.207.129:41427: connect: Connection refused (error 111)
I20260812 21:12:31.913403 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 11963
I20260812 21:12:31.925801 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 11309
2026-08-12T21:12:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2202 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T21:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:31Z Disabled control of system clock
I20260812 21:12:31.965049 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:36721
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:36721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:32.048254 12325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:32.048534 12325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:32.048558 12325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:32.049790 12325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:32.049835 12325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:32.049846 12325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:32.049857 12325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:32.049868 12325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:32.051214 12325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:36721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:36721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:32.051455 12325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:32.051750 12325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:32.056644 12337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:32.056753 12325 server_base.cc:1034] running on GCE node
W20260812 21:12:32.056660 12342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:32.056659 12339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:32.057174 12325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:32.057457 12325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:32.058600 12325 hybrid_clock.cc:648] HybridClock initialized: now 1786569152058587 us; error 26 us; skew 500 ppm
I20260812 21:12:32.059907 12325 webserver.cc:460] Webserver started at http://127.11.207.190:45275/ using document root <none> and password file <none>
I20260812 21:12:32.060129 12325 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:32.060178 12325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:32.060333 12325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:32.061461 12325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "1a9f2f6ba9eb4103b15676680b76f47f"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "943026c6a958ca29abe3417c3ae9ade2"
server_key_iv: "ea3360c368f7387661e0bf105d7db2ba"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.061818 12325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "1a9f2f6ba9eb4103b15676680b76f47f"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "943026c6a958ca29abe3417c3ae9ade2"
server_key_iv: "ea3360c368f7387661e0bf105d7db2ba"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.063060 12325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:12:32.063766 12352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.063949 12325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:32.064040 12325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "1a9f2f6ba9eb4103b15676680b76f47f"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "943026c6a958ca29abe3417c3ae9ade2"
server_key_iv: "ea3360c368f7387661e0bf105d7db2ba"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.064095 12325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:32.075675 12325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:32.075973 12325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:32.076077 12325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:32.079828 12325 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:36721
I20260812 21:12:32.079916 12435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:36721 every 8 connection(s)
I20260812 21:12:32.080212 12325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:32.081234 12436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:32.083607 12436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f: Bootstrap starting.
I20260812 21:12:32.084184 12436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:32.084522 12436 log.cc:824] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:32.085181 12436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f: No bootstrap required, opened a new log
I20260812 21:12:32.086771 12436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36721 } }
I20260812 21:12:32.086910 12436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:32.086930 12436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9f2f6ba9eb4103b15676680b76f47f, State: Initialized, Role: FOLLOWER
I20260812 21:12:32.087072 12436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36721 } }
I20260812 21:12:32.087136 12436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:32.087158 12436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:32.087209 12436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:32.087791 12436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36721 } }
I20260812 21:12:32.087890 12436 leader_election.cc:302] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a9f2f6ba9eb4103b15676680b76f47f; no voters:
I20260812 21:12:32.088063 12436 leader_election.cc:288] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:32.088112 12439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:32.088289 12439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [term 1 LEADER]: Becoming Leader. State: Replica: 1a9f2f6ba9eb4103b15676680b76f47f, State: Running, Role: LEADER
I20260812 21:12:32.088335 12436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:32.088438 12439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36721 } }
I20260812 21:12:32.088822 12442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a9f2f6ba9eb4103b15676680b76f47f. Latest consensus state: current_term: 1 leader_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36721 } } }
I20260812 21:12:32.088904 12441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a9f2f6ba9eb4103b15676680b76f47f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 36721 } } }
I20260812 21:12:32.088915 12442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:32.088975 12441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:32.089138 12453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:32.089540 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12325
I20260812 21:12:32.089653 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:32.089658 12453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:32.089870 12094 external_mini_cluster.cc:1428] Setting key be1a0cec8372e00381c96b5610c387c8
I20260812 21:12:32.094435 12453 catalog_manager.cc:1347] Generated new cluster ID: 0b7fc2a051974ae2843a9ec58797b13e
I20260812 21:12:32.094511 12453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:32.096053 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.090595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.11.207.190:36721)
0812 21:12:32.090800 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.090809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.090845 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:12:32.090962 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.090969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.091049 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.091214 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:32.091449 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.092847 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.094020 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.094526 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.094683 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.095386 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:32.095390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:32.095390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:32.095567 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:32.095601 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:32.095612 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:32.095668 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.095752 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.095832 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260812 21:12:32.120997 12453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:32.121546 12453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:32.128656 12453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f: Generated new TSK 0
I20260812 21:12:32.128906 12453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:32.131407 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36721
--builtin_ntp_servers=127.11.207.148:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:32.219743 12473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:32.219980 12473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:32.220016 12473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:32.221312 12473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:32.221369 12473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:32.221408 12473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:32.222839 12473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:32.223099 12473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:32.223424 12473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:32.228591 12483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:32.228585 12479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:32.228623 12473 server_base.cc:1034] running on GCE node
W20260812 21:12:32.228621 12487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:32.229024 12473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:32.229288 12473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:32.230458 12473 hybrid_clock.cc:648] HybridClock initialized: now 1786569152230441 us; error 52 us; skew 500 ppm
I20260812 21:12:32.231817 12473 webserver.cc:460] Webserver started at http://127.11.207.129:33599/ using document root <none> and password file <none>
I20260812 21:12:32.232043 12473 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:32.232095 12473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:32.232244 12473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:32.233392 12473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "96d12e43406944b5832ce8494483e90a"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "f0129b705e8bf0bca32881752a759717"
server_key_iv: "1fe4da2d5106b1debf66f337163f02f6"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.233744 12473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "96d12e43406944b5832ce8494483e90a"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "f0129b705e8bf0bca32881752a759717"
server_key_iv: "1fe4da2d5106b1debf66f337163f02f6"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.235039 12473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:12:32.235709 12495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.235898 12473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:32.235985 12473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "96d12e43406944b5832ce8494483e90a"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "f0129b705e8bf0bca32881752a759717"
server_key_iv: "1fe4da2d5106b1debf66f337163f02f6"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.236049 12473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:32.248925 12473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:32.249269 12473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:32.249403 12473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:32.249639 12473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:32.249996 12473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:32.250033 12473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.250063 12473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:32.250092 12473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.256116 12473 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:42445
I20260812 21:12:32.256178 12749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:42445 every 8 connection(s)
I20260812 21:12:32.256594 12473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:32.257894 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12473
I20260812 21:12:32.258024 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:12:32.258306 12094 external_mini_cluster.cc:1428] Setting key da38b15a74a1da968902ab5f005fbd3d
I20260812 21:12:32.259260 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36721
--builtin_ntp_servers=127.11.207.148:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:12:32.262004 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.257306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:52613 (local address 127.11.207.190:36721)
0812 21:12:32.257368 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.257371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.257599 (+ 228us) server_negotiation.cc:407] Connection header received
0812 21:12:32.258245 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.258246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.258266 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.258316 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:32.259046 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.259601 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.260642 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.260843 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.260901 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.261717 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:32.261718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:32.261719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:32.261724 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:32.261740 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:32.261745 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:32.261790 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.261889 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.261922 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:12:32.262238 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.257299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:36721 (local address 127.11.207.129:52613)
0812 21:12:32.257491 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.257526 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.258127 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.258405 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.258410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.258421 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:32.258958 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.258963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.259693 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.259693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.260560 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.260563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.260919 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.260920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.261053 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.261585 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:32.261585 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:32.261615 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:32.261639 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:32.261796 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:32.261802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:32.261805 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.261888 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.261996 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260812 21:12:32.262740 12754 heartbeater.cc:347] Connected to a master server at 127.11.207.190:36721
I20260812 21:12:32.262842 12754 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:32.263026 12754 heartbeater.cc:511] Master 127.11.207.190:36721 requested a full tablet report, sending...
I20260812 21:12:32.263621 12378 ts_manager.cc:194] Registered new tserver with Master: 96d12e43406944b5832ce8494483e90a (127.11.207.129:42445)
I20260812 21:12:32.264618 12378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.129:52613
W20260812 21:12:32.345593 12759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:32.345811 12759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:32.345834 12759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:32.346984 12759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:32.347023 12759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:32.347047 12759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:32.348378 12759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:32.348613 12759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:32.348883 12759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:32.353953 12766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:32.353946 12771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:32.353969 12767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:32.354094 12759 server_base.cc:1034] running on GCE node
I20260812 21:12:32.354418 12759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:32.354631 12759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:32.355769 12759 hybrid_clock.cc:648] HybridClock initialized: now 1786569152355748 us; error 42 us; skew 500 ppm
I20260812 21:12:32.357115 12759 webserver.cc:460] Webserver started at http://127.11.207.130:38687/ using document root <none> and password file <none>
I20260812 21:12:32.357373 12759 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:32.357430 12759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:32.357568 12759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:32.358665 12759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "1999a56867b4493592671a8322e71faf"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "73e6d0626e5485c28686a8de52426b46"
server_key_iv: "f567536ea27606e0eb61aa5e1f1ab73c"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.359009 12759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "1999a56867b4493592671a8322e71faf"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "73e6d0626e5485c28686a8de52426b46"
server_key_iv: "f567536ea27606e0eb61aa5e1f1ab73c"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.360265 12759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:32.361016 12782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.361212 12759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:32.361320 12759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "1999a56867b4493592671a8322e71faf"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "73e6d0626e5485c28686a8de52426b46"
server_key_iv: "f567536ea27606e0eb61aa5e1f1ab73c"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.361400 12759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:32.370437 12759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:32.370761 12759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:32.370884 12759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:32.371117 12759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:32.371438 12759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:32.371470 12759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.371492 12759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:32.371515 12759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.377324 12759 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:38105
I20260812 21:12:32.377383 12979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:38105 every 8 connection(s)
I20260812 21:12:32.377709 12759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:12:32.383116 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.378444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:49443 (local address 127.11.207.190:36721)
0812 21:12:32.378509 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.378511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.378733 (+ 222us) server_negotiation.cc:407] Connection header received
0812 21:12:32.379224 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.379225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.379235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.379334 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:32.380036 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.380621 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.381737 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.381943 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.382008 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.382849 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:32.382850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:32.382850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:32.382856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:32.382873 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:32.382877 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:32.382929 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.383023 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.383050 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:12:32.383262 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.378426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:36721 (local address 127.11.207.130:49443)
0812 21:12:32.378606 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.378649 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.379143 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.379390 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.379394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.379403 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:32.379933 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.379938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.380807 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.380809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.381630 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.381632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.382022 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.382023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.382143 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.382675 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:32.382676 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:32.382735 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:32.382762 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:32.382935 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:32.382938 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:32.382941 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.382988 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.383072 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260812 21:12:32.383750 12983 heartbeater.cc:347] Connected to a master server at 127.11.207.190:36721
I20260812 21:12:32.383847 12983 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:32.384030 12983 heartbeater.cc:511] Master 127.11.207.190:36721 requested a full tablet report, sending...
I20260812 21:12:32.384444 12378 ts_manager.cc:194] Registered new tserver with Master: 1999a56867b4493592671a8322e71faf (127.11.207.130:38105)
I20260812 21:12:32.384789 12378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.130:49443
I20260812 21:12:32.385531 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12759
I20260812 21:12:32.385618 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:12:32.385809 12094 external_mini_cluster.cc:1428] Setting key 59ccfa48447eafe8acac82f47868416c
I20260812 21:12:32.386683 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36721
--builtin_ntp_servers=127.11.207.148:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:32.472401 12987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:32.472646 12987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:32.472672 12987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:32.473938 12987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:32.473991 12987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:32.474018 12987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:32.475417 12987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:36721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:32.475679 12987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:32.475988 12987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:32.481065 13004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:32.481127 12998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:32.481128 12999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:32.481356 12987 server_base.cc:1034] running on GCE node
I20260812 21:12:32.481540 12987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:32.481753 12987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:32.482923 12987 hybrid_clock.cc:648] HybridClock initialized: now 1786569152482906 us; error 31 us; skew 500 ppm
I20260812 21:12:32.484362 12987 webserver.cc:460] Webserver started at http://127.11.207.131:42125/ using document root <none> and password file <none>
I20260812 21:12:32.484602 12987 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:32.484651 12987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:32.484786 12987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:32.485888 12987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "166da00b1cea418186af445c17c008d8"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "a7312892c3b6d42872ddd8aaa1fcaa76"
server_key_iv: "1901dddedfdb4f7fb0c8c9cf91708fce"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.486227 12987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "166da00b1cea418186af445c17c008d8"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "a7312892c3b6d42872ddd8aaa1fcaa76"
server_key_iv: "1901dddedfdb4f7fb0c8c9cf91708fce"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.487538 12987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:32.488392 13016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.488562 12987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:32.488649 12987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "166da00b1cea418186af445c17c008d8"
format_stamp: "Formatted at 2026-08-12 21:12:32 on dist-test-slave-lkx2"
server_key: "a7312892c3b6d42872ddd8aaa1fcaa76"
server_key_iv: "1901dddedfdb4f7fb0c8c9cf91708fce"
server_key_version: "encryptionkey@0"
I20260812 21:12:32.488709 12987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:32.511586 12987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:32.511942 12987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:32.512077 12987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:32.512328 12987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:32.512744 12987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:32.512785 12987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.512811 12987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:32.512828 12987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:32.519178 12987 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:42093
I20260812 21:12:32.519222 13244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:42093 every 8 connection(s)
I20260812 21:12:32.519589 12987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:32.522918 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 12987
I20260812 21:12:32.523033 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:12:32.523262 12094 external_mini_cluster.cc:1428] Setting key 8d1b02b8e99cfe0258f7f2808bd6805c
I20260812 21:12:32.525357 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.520461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:34487 (local address 127.11.207.190:36721)
0812 21:12:32.520538 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.520540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.520810 (+ 270us) server_negotiation.cc:407] Connection header received
0812 21:12:32.521323 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.521324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.521336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.521401 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:32.522120 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.522617 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.524059 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.524320 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.524408 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.525146 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:32.525147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:32.525147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:32.525152 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:32.525166 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:32.525170 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:32.525208 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.525265 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.525293 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 21:12:32.525561 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.520430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:36721 (local address 127.11.207.131:34487)
0812 21:12:32.520633 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.520685 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.521205 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.521476 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.521480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.521489 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:32.522047 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.522053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.522720 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.522722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.523982 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.523986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.524420 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.524421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.524541 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.525020 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:32.525020 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:32.525052 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:32.525076 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:32.525214 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:32.525217 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:32.525220 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.525267 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.525354 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260812 21:12:32.526057 13247 heartbeater.cc:347] Connected to a master server at 127.11.207.190:36721
I20260812 21:12:32.526149 13247 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:32.526326 13247 heartbeater.cc:511] Master 127.11.207.190:36721 requested a full tablet report, sending...
I20260812 21:12:32.526628 12378 ts_manager.cc:194] Registered new tserver with Master: 166da00b1cea418186af445c17c008d8 (127.11.207.131:42093)
I20260812 21:12:32.526996 12378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.131:34487
I20260812 21:12:32.535288 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:12:32.539744 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.536796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.11.207.190:36721)
0812 21:12:32.536882 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.536884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.536981 (+ 97us) server_negotiation.cc:407] Connection header received
0812 21:12:32.537028 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.537029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.537043 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.537092 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:32.537281 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.537781 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.538541 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.538775 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.538833 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.539501 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:32.539503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:32.539503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:32.539508 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:32.539524 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:32.539528 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:32.539569 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.539619 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.539648 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 21:12:32.541424 12378 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:12:32.543444 12378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:12:32.550050 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.545797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:42093 (local address 127.0.0.1:47482)
0812 21:12:32.545977 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.546021 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.546101 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.546925 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.546926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.546927 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:32.546956 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.546957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.548100 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.548101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.549048 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.549051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.549741 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.549822 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.549835 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.549909 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.549967 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260812 21:12:32.550064 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.545991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.11.207.131:42093)
0812 21:12:32.546784 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.546788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.546798 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:12:32.546838 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.546839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.546855 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.546931 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:32.547000 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.548042 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549130 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.549664 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549735 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.549848 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.549922 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.549967 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":99,"threads_started":1}
I20260812 21:12:32.550251 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.545802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:38105 (local address 127.0.0.1:34536)
0812 21:12:32.546087 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.546098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.546154 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.546808 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.546809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.546810 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:32.546838 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.546839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.547707 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.547708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.548944 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.548947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.550007 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.550008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.550070 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.550075 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.550119 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.550163 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":110,"threads_started":1}
I20260812 21:12:32.550555 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.546277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.11.207.130:38105)
0812 21:12:32.546464 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.546469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.546479 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:12:32.546526 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.546528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.546543 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.546630 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:32.546895 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.547632 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549034 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.549804 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549877 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.549969 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.550422 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.550471 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260812 21:12:32.550709 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.545802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:42445 (local address 127.0.0.1:43336)
0812 21:12:32.546191 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.546200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.546247 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.546636 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.546639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.546641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:32.546717 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.546719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.548047 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.548049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.549144 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.549146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.550080 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.550080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.550534 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.550540 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.550598 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.550641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":36,"spinlock_wait_cycles":768,"thread_start_us":111,"threads_started":1}
I20260812 21:12:32.550855 13133 tablet_service.cc:1467] Processing CreateTablet for tablet 7fccdaae506c433e8171684d3d110530 (DEFAULT_TABLE table=test-table [id=33b4cb3d66a949a6b7f2c46699021b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:32.551018 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.546088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.11.207.129:42445)
0812 21:12:32.546466 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.546471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.546486 (+ 15us) server_negotiation.cc:407] Connection header received
0812 21:12:32.546533 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.546535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.546550 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.546631 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:32.546818 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.547960 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549221 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.549917 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.549976 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.550050 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.550886 (+ 836us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.550939 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":82,"threads_started":1}
I20260812 21:12:32.551294 13133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fccdaae506c433e8171684d3d110530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:32.551397 12863 tablet_service.cc:1467] Processing CreateTablet for tablet 7fccdaae506c433e8171684d3d110530 (DEFAULT_TABLE table=test-table [id=33b4cb3d66a949a6b7f2c46699021b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:32.551819 12863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fccdaae506c433e8171684d3d110530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:32.551848 12596 tablet_service.cc:1467] Processing CreateTablet for tablet 7fccdaae506c433e8171684d3d110530 (DEFAULT_TABLE table=test-table [id=33b4cb3d66a949a6b7f2c46699021b4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:32.552191 12596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fccdaae506c433e8171684d3d110530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:32.554104 13272 tablet_bootstrap.cc:492] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Bootstrap starting.
I20260812 21:12:32.554831 13272 tablet_bootstrap.cc:654] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:32.555127 13272 log.cc:824] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:32.555397 13273 tablet_bootstrap.cc:492] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Bootstrap starting.
I20260812 21:12:32.555941 13277 tablet_bootstrap.cc:492] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: Bootstrap starting.
I20260812 21:12:32.556015 13272 tablet_bootstrap.cc:492] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: No bootstrap required, opened a new log
I20260812 21:12:32.556087 13272 ts_tablet_manager.cc:1397] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:12:32.556329 13273 tablet_bootstrap.cc:654] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:32.556635 13277 tablet_bootstrap.cc:654] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:32.556778 13273 log.cc:824] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:32.556943 13277 log.cc:824] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:32.557607 13273 tablet_bootstrap.cc:492] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: No bootstrap required, opened a new log
I20260812 21:12:32.557672 13273 ts_tablet_manager.cc:1397] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 21:12:32.557699 13277 tablet_bootstrap.cc:492] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: No bootstrap required, opened a new log
I20260812 21:12:32.557776 13277 ts_tablet_manager.cc:1397] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:12:32.558620 13272 raft_consensus.cc:348] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.558821 13272 raft_consensus.cc:374] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:32.558859 13272 raft_consensus.cc:729] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166da00b1cea418186af445c17c008d8, State: Initialized, Role: FOLLOWER
I20260812 21:12:32.559005 13272 consensus_queue.cc:260] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.559293 13272 ts_tablet_manager.cc:1428] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 21:12:32.559791 13273 raft_consensus.cc:348] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.559917 13273 raft_consensus.cc:374] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:32.559950 13273 raft_consensus.cc:729] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1999a56867b4493592671a8322e71faf, State: Initialized, Role: FOLLOWER
I20260812 21:12:32.560061 13273 consensus_queue.cc:260] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.560314 13273 ts_tablet_manager.cc:1428] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:12:32.560376 12983 heartbeater.cc:503] Master 127.11.207.190:36721 was elected leader, sending a full tablet report...
I20260812 21:12:32.560461 13247 heartbeater.cc:503] Master 127.11.207.190:36721 was elected leader, sending a full tablet report...
I20260812 21:12:32.560520 13277 raft_consensus.cc:348] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.560675 13277 raft_consensus.cc:374] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:32.560716 13277 raft_consensus.cc:729] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96d12e43406944b5832ce8494483e90a, State: Initialized, Role: FOLLOWER
I20260812 21:12:32.560823 13277 consensus_queue.cc:260] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.561084 13277 ts_tablet_manager.cc:1428] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 21:12:32.561123 12754 heartbeater.cc:503] Master 127.11.207.190:36721 was elected leader, sending a full tablet report...
I20260812 21:12:32.615185 13288 raft_consensus.cc:482] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:32.615337 13288 raft_consensus.cc:504] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.615784 13288 leader_election.cc:288] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96d12e43406944b5832ce8494483e90a (127.11.207.129:42445), 166da00b1cea418186af445c17c008d8 (127.11.207.131:42093)
I20260812 21:12:32.618845 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.616078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:34571 (local address 127.11.207.129:42445)
0812 21:12:32.616124 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.616126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.616360 (+ 234us) server_negotiation.cc:407] Connection header received
0812 21:12:32.616417 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.616418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.616430 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.616475 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:32.616622 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.617109 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.618102 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.618497 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.618563 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.618571 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.618709 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.618778 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:12:32.618855 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.616043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:44807 (local address 127.11.207.131:42093)
0812 21:12:32.616087 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.616089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.616101 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:12:32.616171 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.616172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.616182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.616254 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:32.616521 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.617034 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.618123 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.618503 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.618567 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.618576 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.618715 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.618760 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:12:32.618855 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.615967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:42093 (local address 127.11.207.130:44807)
0812 21:12:32.616033 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.616047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.616103 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.616312 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.616313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.616316 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:32.616432 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.616434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.617116 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.617117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.618025 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.618028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.618582 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.618583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.618653 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.618656 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.618700 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.618747 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":1}
I20260812 21:12:32.618856 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.615973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:42445 (local address 127.11.207.130:34571)
0812 21:12:32.616274 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.616285 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:32.616363 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:32.616486 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:32.616487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:32.616488 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:32.616560 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.616562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.617188 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.617189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.618012 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:32.618015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.618575 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.618576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:32.618649 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.618653 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:32.618697 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:12:32.618741 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":80,"thread_start_us":122,"threads_started":1}
I20260812 21:12:32.619242 13169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fccdaae506c433e8171684d3d110530" candidate_uuid: "1999a56867b4493592671a8322e71faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166da00b1cea418186af445c17c008d8" is_pre_election: true
I20260812 21:12:32.619325 12638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fccdaae506c433e8171684d3d110530" candidate_uuid: "1999a56867b4493592671a8322e71faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96d12e43406944b5832ce8494483e90a" is_pre_election: true
I20260812 21:12:32.619413 13169 raft_consensus.cc:2454] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1999a56867b4493592671a8322e71faf in term 0.
I20260812 21:12:32.619488 12638 raft_consensus.cc:2454] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1999a56867b4493592671a8322e71faf in term 0.
I20260812 21:12:32.619693 12788 leader_election.cc:302] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166da00b1cea418186af445c17c008d8, 1999a56867b4493592671a8322e71faf; no voters:
I20260812 21:12:32.619832 13288 raft_consensus.cc:2781] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:12:32.619901 13288 raft_consensus.cc:482] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:12:32.619923 13288 raft_consensus.cc:3037] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:32.620692 13288 raft_consensus.cc:504] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.620832 13288 leader_election.cc:288] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [CANDIDATE]: Term 1 election: Requested vote from peers 96d12e43406944b5832ce8494483e90a (127.11.207.129:42445), 166da00b1cea418186af445c17c008d8 (127.11.207.131:42093)
I20260812 21:12:32.621035 12638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fccdaae506c433e8171684d3d110530" candidate_uuid: "1999a56867b4493592671a8322e71faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96d12e43406944b5832ce8494483e90a"
I20260812 21:12:32.621029 13169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fccdaae506c433e8171684d3d110530" candidate_uuid: "1999a56867b4493592671a8322e71faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166da00b1cea418186af445c17c008d8"
I20260812 21:12:32.621129 13169 raft_consensus.cc:3037] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:32.621127 12638 raft_consensus.cc:3037] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:32.621956 12638 raft_consensus.cc:2454] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1999a56867b4493592671a8322e71faf in term 1.
I20260812 21:12:32.621999 13169 raft_consensus.cc:2454] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1999a56867b4493592671a8322e71faf in term 1.
I20260812 21:12:32.622153 12784 leader_election.cc:302] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1999a56867b4493592671a8322e71faf, 96d12e43406944b5832ce8494483e90a; no voters:
I20260812 21:12:32.622258 13288 raft_consensus.cc:2781] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:32.622414 13288 raft_consensus.cc:686] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 1 LEADER]: Becoming Leader. State: Replica: 1999a56867b4493592671a8322e71faf, State: Running, Role: LEADER
I20260812 21:12:32.622506 13288 consensus_queue.cc:237] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } }
I20260812 21:12:32.623332 12378 catalog_manager.cc:5495] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1999a56867b4493592671a8322e71faf (127.11.207.130). New cstate: current_term: 1 leader_uuid: "1999a56867b4493592671a8322e71faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1999a56867b4493592671a8322e71faf" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 38105 } health_report { overall_health: HEALTHY } } }
W20260812 21:12:32.628263 12984 tablet.cc:2369] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:12:32.644842 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.642275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.11.207.130:38105)
0812 21:12:32.642339 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:32.642341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:32.642348 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:12:32.642391 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:32.642392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:32.642412 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:32.642472 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:12:32.642575 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.643050 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.643792 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:32.643997 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:32.644061 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:32.644336 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:12:32.644514 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:12:32.644595 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:32.644706 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 21:12:32.644729 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:12:32.646811 12638 raft_consensus.cc:1261] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 1 FOLLOWER]: Refusing update from remote peer 1999a56867b4493592671a8322e71faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:32.646829 13169 raft_consensus.cc:1261] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 1 FOLLOWER]: Refusing update from remote peer 1999a56867b4493592671a8322e71faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:32.647176 13293 consensus_queue.cc:1035] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "166da00b1cea418186af445c17c008d8" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:32.647311 13288 consensus_queue.cc:1035] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "96d12e43406944b5832ce8494483e90a" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:32.650377 13296 mvcc.cc:205] Tried to move back new op lower bound from 7317787249238536192 to 7317787249142439936. Current Snapshot: MvccSnapshot[applied={T|T < 7317787249238536192 or (T in {7317787249238536192})}]
I20260812 21:12:32.650524 13298 mvcc.cc:205] Tried to move back new op lower bound from 7317787249238536192 to 7317787249142439936. Current Snapshot: MvccSnapshot[applied={T|T < 7317787249238536192}]
I20260812 21:12:32.650635 13297 mvcc.cc:205] Tried to move back new op lower bound from 7317787249238536192 to 7317787249142439936. Current Snapshot: MvccSnapshot[applied={T|T < 7317787249238536192}]
I20260812 21:12:32.652060 12377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:32.652171 12377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:32.653015 12377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a9f2f6ba9eb4103b15676680b76f47f: Sending DeleteTablet for 3 replicas of tablet 7fccdaae506c433e8171684d3d110530
I20260812 21:12:32.653370 12863 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fccdaae506c433e8171684d3d110530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:32 UTC) from {username='slave'} at 127.0.0.1:34536
I20260812 21:12:32.653350 12596 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fccdaae506c433e8171684d3d110530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:32 UTC) from {username='slave'} at 127.0.0.1:43336
I20260812 21:12:32.653402 13133 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fccdaae506c433e8171684d3d110530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:32 UTC) from {username='slave'} at 127.0.0.1:47482
I20260812 21:12:32.653743 13320 tablet_replica.cc:330] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a: stopping tablet replica
I20260812 21:12:32.653851 13320 raft_consensus.cc:2229] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:32.653960 13320 raft_consensus.cc:2258] T 7fccdaae506c433e8171684d3d110530 P 96d12e43406944b5832ce8494483e90a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:32.654057 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12473
I20260812 21:12:32.654848 13318 tablet_replica.cc:330] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: stopping tablet replica
I20260812 21:12:32.654990 13318 raft_consensus.cc:2229] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:12:32.655205 13318 raft_consensus.cc:2258] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:32.654584 13322 tablet_replica.cc:330] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: stopping tablet replica
I20260812 21:12:32.657769 13322 raft_consensus.cc:2229] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:32.657889 13322 raft_consensus.cc:2258] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:32.658051 13318 ts_tablet_manager.cc:1905] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:32.658337 13322 ts_tablet_manager.cc:1905] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:32.660231 13322 ts_tablet_manager.cc:1918] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:32.660247 13318 ts_tablet_manager.cc:1918] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:32.660331 13322 log.cc:1196] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/7fccdaae506c433e8171684d3d110530
I20260812 21:12:32.660487 13318 log.cc:1196] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/7fccdaae506c433e8171684d3d110530
I20260812 21:12:32.660648 13322 ts_tablet_manager.cc:1939] T 7fccdaae506c433e8171684d3d110530 P 166da00b1cea418186af445c17c008d8: Deleting consensus metadata
I20260812 21:12:32.661139 12356 catalog_manager.cc:4841] TS 166da00b1cea418186af445c17c008d8 (127.11.207.131:42093): tablet 7fccdaae506c433e8171684d3d110530 (table test-table [id=33b4cb3d66a949a6b7f2c46699021b4e]) successfully deleted
I20260812 21:12:32.660969 13318 ts_tablet_manager.cc:1939] T 7fccdaae506c433e8171684d3d110530 P 1999a56867b4493592671a8322e71faf: Deleting consensus metadata
I20260812 21:12:32.662577 12357 catalog_manager.cc:4841] TS 1999a56867b4493592671a8322e71faf (127.11.207.130:38105): tablet 7fccdaae506c433e8171684d3d110530 (table test-table [id=33b4cb3d66a949a6b7f2c46699021b4e]) successfully deleted
W20260812 21:12:32.676796 12353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:42445 (error 108)
I20260812 21:12:32.677094 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12759
I20260812 21:12:32.677531 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:32.677358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:42445 (local address 127.0.0.1:43352)
0812 21:12:32.677424 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:32.677491 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:12:32.680258 12353 catalog_manager.cc:4584] TS 96d12e43406944b5832ce8494483e90a (127.11.207.129:42445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7fccdaae506c433e8171684d3d110530: Network error: Client connection negotiation failed: client connection to 127.11.207.129:42445: connect: Connection refused (error 111)
I20260812 21:12:32.690718 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12987
I20260812 21:12:32.699846 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 12325
2026-08-12T21:12:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (771 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:32 dist-test-slave-lkx2 krb5kdc[13327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:32 dist-test-slave-lkx2 krb5kdc[13327](info): set up 2 sockets
Aug 12 21:12:32 dist-test-slave-lkx2 krb5kdc[13327](info): commencing operation
krb5kdc: starting...
W20260812 21:12:33.916983 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:12:34.041273 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43397
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:34.124650 13358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:34.124922 13358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:34.124955 13358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:34.126209 13358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:34.126263 13358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:34.126281 13358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:34.126298 13358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:34.126312 13358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:34.127942 13358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:34.128223 13358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:34.128477 13358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:34.133456 13371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:34.133450 13363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:34.133572 13358 server_base.cc:1034] running on GCE node
W20260812 21:12:34.133484 13364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:34.133982 13358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:34.134285 13358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:34.135435 13358 hybrid_clock.cc:648] HybridClock initialized: now 1786569154135418 us; error 35 us; skew 500 ppm
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:34.138044 13358 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:12:34.138517 13358 webserver.cc:460] Webserver started at http://127.11.207.190:40511/ using document root <none> and password file <none>
I20260812 21:12:34.138725 13358 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:34.138765 13358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:34.138861 13358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:34.139925 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "8b5a53d298b94955bb93b9f29d1b105b"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "64866e4d0749fc851d3c446ee545add4"
server_key_iv: "fc99e382578d660b0955ed0e5402bba4"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.140231 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "8b5a53d298b94955bb93b9f29d1b105b"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "64866e4d0749fc851d3c446ee545add4"
server_key_iv: "fc99e382578d660b0955ed0e5402bba4"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.141584 13358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:34.142292 13384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.142450 13358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:34.142537 13358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "8b5a53d298b94955bb93b9f29d1b105b"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "64866e4d0749fc851d3c446ee545add4"
server_key_iv: "fc99e382578d660b0955ed0e5402bba4"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.142607 13358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:34.175166 13358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:34.177528 13358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:34.177652 13358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:34.181694 13358 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43397
I20260812 21:12:34.181749 13480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43397 every 8 connection(s)
I20260812 21:12:34.182139 13358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:34.183128 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:34.185564 13482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b: Bootstrap starting.
I20260812 21:12:34.186147 13482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:34.186426 13482 log.cc:824] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:34.187161 13482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b: No bootstrap required, opened a new log
I20260812 21:12:34.187237 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13358
I20260812 21:12:34.187350 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:34.187511 12094 external_mini_cluster.cc:1428] Setting key 4eac44672d63d6af37166e44cf6f87fe
I20260812 21:12:34.189101 13482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a53d298b94955bb93b9f29d1b105b" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43397 } }
I20260812 21:12:34.189248 13482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:34.189283 13482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5a53d298b94955bb93b9f29d1b105b, State: Initialized, Role: FOLLOWER
I20260812 21:12:34.189433 13482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a53d298b94955bb93b9f29d1b105b" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43397 } }
I20260812 21:12:34.189517 13482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:34.189551 13482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:34.189601 13482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:34.190577 13482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a53d298b94955bb93b9f29d1b105b" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43397 } }
I20260812 21:12:34.190681 13482 leader_election.cc:302] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b5a53d298b94955bb93b9f29d1b105b; no voters:
I20260812 21:12:34.190856 13482 leader_election.cc:288] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:34.190891 13493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:34.191001 13493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [term 1 LEADER]: Becoming Leader. State: Replica: 8b5a53d298b94955bb93b9f29d1b105b, State: Running, Role: LEADER
I20260812 21:12:34.191131 13482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:34.191154 13493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a53d298b94955bb93b9f29d1b105b" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43397 } }
I20260812 21:12:34.191505 13493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b5a53d298b94955bb93b9f29d1b105b. Latest consensus state: current_term: 1 leader_uuid: "8b5a53d298b94955bb93b9f29d1b105b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a53d298b94955bb93b9f29d1b105b" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43397 } } }
I20260812 21:12:34.191615 13493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:34.192399 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b5a53d298b94955bb93b9f29d1b105b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a53d298b94955bb93b9f29d1b105b" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43397 } } }
I20260812 21:12:34.192478 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:34.192608 13508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:34.193215 13508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:34.194941 13508 catalog_manager.cc:1347] Generated new cluster ID: f9603480ce5741f9b21ccbbe15474d3f
I20260812 21:12:34.194993 13508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:34.199271 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.188019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.11.207.190:43397)
0812 21:12:34.188214 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.188221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.188248 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:12:34.189080 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.189089 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.189156 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.189298 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:34.190036 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.191081 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.192067 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.192686 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.192793 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.195626 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:34.195630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:34.195630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:34.195643 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:34.197582 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.197976 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.197979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.197980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.198119 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.198314 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.198315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.198315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.198655 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:34.198817 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.198888 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.199033 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260812 21:12:34.207885 13508 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:34.208492 13508 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:34.211576 13508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b: Generated new TSK 0
I20260812 21:12:34.211766 13508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:12:34.245606 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43397
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:34.359122 13538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:34.359342 13538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:34.359368 13538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:34.360579 13538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:34.360622 13538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:34.360651 13538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:34.362231 13538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:34.362465 13538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:34.362705 13538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:34.367734 13547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:34.367738 13548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:34.367770 13550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:34.368242 13538 server_base.cc:1034] running on GCE node
I20260812 21:12:34.368475 13538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:34.368775 13538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:34.369927 13538 hybrid_clock.cc:648] HybridClock initialized: now 1786569154369914 us; error 35 us; skew 500 ppm
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:34.372613 13538 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:12:34.373096 13538 webserver.cc:460] Webserver started at http://127.11.207.129:45419/ using document root <none> and password file <none>
I20260812 21:12:34.373313 13538 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:34.373356 13538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:34.373466 13538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:34.374490 13538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "cae09d773cde4a389802f3ab2deb96f2"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "852264d2d498f28f2323d94a9a87530a"
server_key_iv: "8377012100e708b33465b0aa4706c4fd"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.374877 13538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "cae09d773cde4a389802f3ab2deb96f2"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "852264d2d498f28f2323d94a9a87530a"
server_key_iv: "8377012100e708b33465b0aa4706c4fd"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.376397 13538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:12:34.377213 13563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.377401 13538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:34.377493 13538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "cae09d773cde4a389802f3ab2deb96f2"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "852264d2d498f28f2323d94a9a87530a"
server_key_iv: "8377012100e708b33465b0aa4706c4fd"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.377553 13538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:34.401382 13538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:34.403337 13538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:34.403455 13538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:34.403661 13538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:34.403931 13538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:34.403963 13538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.403985 13538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:34.403997 13538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.409883 13538 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:39381
I20260812 21:12:34.409960 13774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:39381 every 8 connection(s)
I20260812 21:12:34.410281 13538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:34.412464 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13538
I20260812 21:12:34.412555 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:12:34.412739 12094 external_mini_cluster.cc:1428] Setting key af084ef8feb2d8a50909f360b0ad7920
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:34.420598 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.411060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:41091 (local address 127.11.207.190:43397)
0812 21:12:34.411112 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.411113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.411360 (+ 247us) server_negotiation.cc:407] Connection header received
0812 21:12:34.411874 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.411875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.411890 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.411951 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:34.413131 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.413793 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.414793 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.415077 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.415151 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.417727 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:34.417729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:34.417729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:34.417738 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:34.419202 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.419624 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.419625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.419626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.419681 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.419914 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.419915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.419915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.420037 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:34.420095 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.420402 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.420501 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:12:34.420826 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.411029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43397 (local address 127.11.207.129:41091)
0812 21:12:34.411216 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.411261 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.411786 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.412022 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.412026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.412750 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:34.413043 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.413051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.413887 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.413888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.414712 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.414714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.415177 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.415178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.415340 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.415892 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:34.415904 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:34.417594 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:34.419295 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:34.419304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:34.419306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:34.419551 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:34.419737 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:34.419738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:34.419739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:34.419855 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:34.420093 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:34.420101 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:34.420249 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.420371 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.420551 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 21:12:34.421353 13776 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43397
I20260812 21:12:34.421471 13776 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:34.421698 13776 heartbeater.cc:511] Master 127.11.207.190:43397 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
I20260812 21:12:34.422678 13412 ts_manager.cc:194] Registered new tserver with Master: cae09d773cde4a389802f3ab2deb96f2 (127.11.207.129:39381)
I20260812 21:12:34.423370 13412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:41091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:12:34.441054 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43397
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:34.524964 13783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:34.525180 13783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:34.525204 13783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:34.526335 13783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:34.526374 13783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:34.526399 13783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:34.527902 13783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:34.528111 13783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:34.528298 13783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:34.533321 13799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:34.533317 13795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:34.533326 13796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:34.533598 13783 server_base.cc:1034] running on GCE node
I20260812 21:12:34.533804 13783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:34.534022 13783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:34.535176 13783 hybrid_clock.cc:648] HybridClock initialized: now 1786569154535160 us; error 27 us; skew 500 ppm
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:34.537914 13783 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:12:34.538326 13783 webserver.cc:460] Webserver started at http://127.11.207.130:34175/ using document root <none> and password file <none>
I20260812 21:12:34.538529 13783 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:34.538571 13783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:34.538704 13783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:34.539764 13783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "c888c18b7b9e4b5198d74efa5f0fdd99"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "2ade9e6729b0e9533a2805668c1b76d1"
server_key_iv: "9f372c884f415c43c921525417b2e2d1"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.540073 13783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "c888c18b7b9e4b5198d74efa5f0fdd99"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "2ade9e6729b0e9533a2805668c1b76d1"
server_key_iv: "9f372c884f415c43c921525417b2e2d1"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.541405 13783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:12:34.542243 13814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.542433 13783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:34.542526 13783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "c888c18b7b9e4b5198d74efa5f0fdd99"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "2ade9e6729b0e9533a2805668c1b76d1"
server_key_iv: "9f372c884f415c43c921525417b2e2d1"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.542587 13783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:34.553478 13783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:34.555527 13783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:34.555640 13783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:34.555871 13783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:34.556146 13783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:34.556178 13783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.556200 13783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:34.556218 13783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.561682 13783 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:36109
I20260812 21:12:34.561746 14021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:36109 every 8 connection(s)
I20260812 21:12:34.562037 13783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:12:34.566931 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 13783
I20260812 21:12:34.567047 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:12:34.567278 12094 external_mini_cluster.cc:1428] Setting key 00f4b44d039ac37910022f4ca6315cfb
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:34.571811 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.562746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:38089 (local address 127.11.207.190:43397)
0812 21:12:34.562805 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.562807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.562979 (+ 172us) server_negotiation.cc:407] Connection header received
0812 21:12:34.563451 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.563452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.563462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.563523 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:34.564535 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.565037 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.566067 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.566271 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.566336 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.568776 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:34.568778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:34.568778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:34.568787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:34.570304 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.570798 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.570799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.570799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.570858 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.571140 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.571141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.571142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.571274 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:34.571349 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.571625 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.571723 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:12:34.571962 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.562692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43397 (local address 127.11.207.130:38089)
0812 21:12:34.562852 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.562895 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.563374 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.563591 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.563595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.564183 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:34.564440 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.564448 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.565126 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.565127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.565959 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.565961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.566355 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.566356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.566464 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.567034 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:34.567045 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:34.568674 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:34.570394 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:34.570403 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:34.570405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:34.570719 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:34.570912 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:34.570913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:34.570913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:34.571051 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:34.571344 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:34.571351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:34.571489 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.571585 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.571756 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260812 21:12:34.572494 14023 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43397
I20260812 21:12:34.572585 14023 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:34.572762 14023 heartbeater.cc:511] Master 127.11.207.190:43397 requested a full tablet report, sending...
I20260812 21:12:34.573173 13412 ts_manager.cc:194] Registered new tserver with Master: c888c18b7b9e4b5198d74efa5f0fdd99 (127.11.207.130:36109)
I20260812 21:12:34.573566 13412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:38089
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:12:34.595319 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43397
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:34.680397 14036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:34.680613 14036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:34.680644 14036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:34.681900 14036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:34.681944 14036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:34.681970 14036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:34.683415 14036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:43397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:34.683648 14036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:34.683866 14036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:34.688920 14045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:34.689002 14036 server_base.cc:1034] running on GCE node
W20260812 21:12:34.689005 14050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:34.688939 14048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:34.689386 14036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:34.689610 14036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:34.690742 14036 hybrid_clock.cc:648] HybridClock initialized: now 1786569154690729 us; error 33 us; skew 500 ppm
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:34.693367 14036 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:12:34.693843 14036 webserver.cc:460] Webserver started at http://127.11.207.131:36421/ using document root <none> and password file <none>
I20260812 21:12:34.694059 14036 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:34.694108 14036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:34.694237 14036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:34.695276 14036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "ed2ba21c83be40ba92aa04257677bf6b"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "5c40e6654f6fa99380ccf17e149a7e3e"
server_key_iv: "7f8f57ffa5eedf04dfc261a05b9c8876"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.695621 14036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "ed2ba21c83be40ba92aa04257677bf6b"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "5c40e6654f6fa99380ccf17e149a7e3e"
server_key_iv: "7f8f57ffa5eedf04dfc261a05b9c8876"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.696964 14036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:34.697685 14068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.697835 14036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:34.697921 14036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "ed2ba21c83be40ba92aa04257677bf6b"
format_stamp: "Formatted at 2026-08-12 21:12:34 on dist-test-slave-lkx2"
server_key: "5c40e6654f6fa99380ccf17e149a7e3e"
server_key_iv: "7f8f57ffa5eedf04dfc261a05b9c8876"
server_key_version: "encryptionkey@0"
I20260812 21:12:34.697988 14036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:34.709833 14036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:34.712141 14036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:34.712296 14036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:34.712566 14036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:34.712877 14036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:34.712911 14036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.712942 14036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:34.712965 14036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:34.718998 14036 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:46437
I20260812 21:12:34.719056 14264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:46437 every 8 connection(s)
I20260812 21:12:34.719388 14036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:34.721410 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14036
I20260812 21:12:34.721488 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:12:34.721657 12094 external_mini_cluster.cc:1428] Setting key 766acc4f654583b9aae6db543eb05414
Aug 12 21:12:34 dist-test-slave-lkx2 krb5kdc[13327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569154, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:34.729359 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.720148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:53213 (local address 127.11.207.190:43397)
0812 21:12:34.720233 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.720236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.720448 (+ 212us) server_negotiation.cc:407] Connection header received
0812 21:12:34.721003 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.721004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.721015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.721067 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:34.722215 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.722704 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.723896 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.724100 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.724158 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.726491 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:34.726492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:34.726492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:34.726500 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:34.727877 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.728286 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.728287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.728287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.728361 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.728642 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.728643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.728643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.728797 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:34.728856 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.729201 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.729288 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 21:12:34.729497 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.720117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:43397 (local address 127.11.207.131:53213)
0812 21:12:34.720286 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.720328 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.720903 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.721132 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.721136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.721820 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:34.722130 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.722138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.722812 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.722813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.723818 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.723821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.724201 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.724202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.724326 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.724884 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:34.724894 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:34.726382 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:34.727963 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:34.727972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:34.727975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:34.728225 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:34.728446 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:34.728447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:34.728448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:34.728580 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:34.728887 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:34.728902 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:34.729041 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.729143 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.729290 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260812 21:12:34.730137 14266 heartbeater.cc:347] Connected to a master server at 127.11.207.190:43397
I20260812 21:12:34.730249 14266 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:34.730465 14266 heartbeater.cc:511] Master 127.11.207.190:43397 requested a full tablet report, sending...
I20260812 21:12:34.730978 13412 ts_manager.cc:194] Registered new tserver with Master: ed2ba21c83be40ba92aa04257677bf6b (127.11.207.131:46437)
I20260812 21:12:34.731359 13412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:53213
I20260812 21:12:34.734326 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:12:34.741825 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.735677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.11.207.190:43397)
0812 21:12:34.735766 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.735768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.735859 (+ 91us) server_negotiation.cc:407] Connection header received
0812 21:12:34.735923 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.735924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.735934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.735980 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:34.736577 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.737071 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.737778 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.737966 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.738019 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.739372 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:34.739374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:34.739374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:34.739383 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:34.740648 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.741038 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.741039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.741039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.741093 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:34.741288 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:34.741289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:34.741289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:34.741432 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:34.741531 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.741667 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.741731 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 21:12:34.743420 13412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:12:34.745401 13412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:12:34.752110 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.748114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:36109 (local address 127.0.0.1:57554)
0812 21:12:34.748747 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.748757 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.748798 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.749067 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.749068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.749070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:34.749110 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.749111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.750290 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.750291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.751035 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.751038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.751899 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.751952 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.751955 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.752006 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.752037 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":69,"threads_started":1}
I20260812 21:12:34.752151 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.748560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57554 (local address 127.11.207.130:36109)
0812 21:12:34.748913 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.748918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.748927 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:12:34.748964 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.748966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.748981 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.749054 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:34.749167 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.750200 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751118 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.751828 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751905 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.751963 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.752021 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.752061 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":78,"threads_started":1}
I20260812 21:12:34.752110 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.747894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:39381 (local address 127.0.0.1:34920)
0812 21:12:34.748064 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.748106 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.748193 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.748424 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.748428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.748430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:34.748508 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.748510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.749620 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.749621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.751067 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.751070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.751780 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.751873 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.751890 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.751957 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.752013 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260812 21:12:34.752429 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.748029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.11.207.129:39381)
0812 21:12:34.748208 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.748213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.748228 (+ 15us) server_negotiation.cc:407] Connection header received
0812 21:12:34.748297 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.748299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.748317 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.748422 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:34.748582 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.749521 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751163 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.751698 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751774 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.751838 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.752162 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.752210 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 21:12:34.752792 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.748260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.11.207.131:46437)
0812 21:12:34.748445 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.748449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.748461 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:12:34.748502 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.748505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.748521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.748687 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:34.748915 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.750006 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751237 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.751900 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.751959 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.752005 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.752667 (+ 662us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.752714 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260812 21:12:34.752979 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.747901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:46437 (local address 127.0.0.1:51856)
0812 21:12:34.748209 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.748220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.748277 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.748780 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.748781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.748783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:34.748836 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.748839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.750117 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.750118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.751162 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.751163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.752040 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.752041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.752099 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.752103 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.752149 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.752914 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":22,"thread_start_us":103,"threads_started":1}
I20260812 21:12:34.753207 13895 tablet_service.cc:1467] Processing CreateTablet for tablet 90c7242138824986931440d6e9fc0ae5 (DEFAULT_TABLE table=test-table [id=2282a5e3b273468a94b74921e6098b69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:34.753199 13660 tablet_service.cc:1467] Processing CreateTablet for tablet 90c7242138824986931440d6e9fc0ae5 (DEFAULT_TABLE table=test-table [id=2282a5e3b273468a94b74921e6098b69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:34.753523 13895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c7242138824986931440d6e9fc0ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:34.753536 13660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c7242138824986931440d6e9fc0ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:34.756104 14302 tablet_bootstrap.cc:492] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Bootstrap starting.
I20260812 21:12:34.756526 14305 tablet_bootstrap.cc:492] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Bootstrap starting.
I20260812 21:12:34.756862 14302 tablet_bootstrap.cc:654] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:34.757113 14305 tablet_bootstrap.cc:654] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:34.757050 14150 tablet_service.cc:1467] Processing CreateTablet for tablet 90c7242138824986931440d6e9fc0ae5 (DEFAULT_TABLE table=test-table [id=2282a5e3b273468a94b74921e6098b69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:34.757411 14150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c7242138824986931440d6e9fc0ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:34.757428 14302 log.cc:824] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:34.757642 14305 log.cc:824] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:34.758198 14302 tablet_bootstrap.cc:492] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: No bootstrap required, opened a new log
I20260812 21:12:34.758284 14302 ts_tablet_manager.cc:1397] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:12:34.758424 14305 tablet_bootstrap.cc:492] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: No bootstrap required, opened a new log
I20260812 21:12:34.758487 14305 ts_tablet_manager.cc:1397] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:12:34.760275 14308 tablet_bootstrap.cc:492] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Bootstrap starting.
I20260812 21:12:34.760231 14302 raft_consensus.cc:348] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.760468 14302 raft_consensus.cc:374] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:34.760512 14302 raft_consensus.cc:729] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c888c18b7b9e4b5198d74efa5f0fdd99, State: Initialized, Role: FOLLOWER
I20260812 21:12:34.760630 14302 consensus_queue.cc:260] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.760674 14305 raft_consensus.cc:348] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.760854 14305 raft_consensus.cc:374] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:34.760901 14305 raft_consensus.cc:729] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae09d773cde4a389802f3ab2deb96f2, State: Initialized, Role: FOLLOWER
I20260812 21:12:34.760928 14302 ts_tablet_manager.cc:1428] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:12:34.761058 14023 heartbeater.cc:503] Master 127.11.207.190:43397 was elected leader, sending a full tablet report...
I20260812 21:12:34.761039 14305 consensus_queue.cc:260] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.761211 14308 tablet_bootstrap.cc:654] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:34.761303 14305 ts_tablet_manager.cc:1428] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:12:34.761341 13776 heartbeater.cc:503] Master 127.11.207.190:43397 was elected leader, sending a full tablet report...
I20260812 21:12:34.761816 14308 log.cc:824] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:34.762662 14308 tablet_bootstrap.cc:492] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: No bootstrap required, opened a new log
I20260812 21:12:34.762734 14308 ts_tablet_manager.cc:1397] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:12:34.764132 14308 raft_consensus.cc:348] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.764271 14308 raft_consensus.cc:374] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:34.764292 14308 raft_consensus.cc:729] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2ba21c83be40ba92aa04257677bf6b, State: Initialized, Role: FOLLOWER
I20260812 21:12:34.764421 14308 consensus_queue.cc:260] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.764683 14308 ts_tablet_manager.cc:1428] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:12:34.764734 14266 heartbeater.cc:503] Master 127.11.207.190:43397 was elected leader, sending a full tablet report...
W20260812 21:12:34.812676 14026 tablet.cc:2369] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:12:34.911095 13777 tablet.cc:2369] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:12:34.969997 14269 tablet.cc:2369] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:12:34.982475 14309 raft_consensus.cc:482] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:34.982625 14309 raft_consensus.cc:504] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.983016 14309 leader_election.cc:288] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cae09d773cde4a389802f3ab2deb96f2 (127.11.207.129:39381), ed2ba21c83be40ba92aa04257677bf6b (127.11.207.131:46437)
I20260812 21:12:34.985759 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.983262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:60941 (local address 127.11.207.129:39381)
0812 21:12:34.983306 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.983308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.983320 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:12:34.983400 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.983401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.983410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.983454 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:34.983607 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.984101 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.985077 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.985432 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.985496 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.985506 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.985654 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.985692 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:12:34.985759 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.983208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:39381 (local address 127.11.207.130:60941)
0812 21:12:34.983269 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.983281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.983330 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.983468 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.983469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.983470 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:34.983545 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.983547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.984196 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.984197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.984987 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.984989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.985502 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.985503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.985585 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.985588 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.985633 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.985675 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 21:12:34.985854 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.983285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:42933 (local address 127.11.207.131:46437)
0812 21:12:34.983355 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:34.983357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:34.983497 (+ 140us) server_negotiation.cc:407] Connection header received
0812 21:12:34.983544 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:34.983545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:34.983554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:34.983597 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:34.983744 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.984207 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.985155 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.985531 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.985596 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.985604 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:34.985747 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:12:34.985784 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 21:12:34.985879 14316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:34.983226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:46437 (local address 127.11.207.130:42933)
0812 21:12:34.983422 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:34.983431 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:34.983490 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:34.983604 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:34.983605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:34.983607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:34.983676 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.983677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.984279 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.984280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.985064 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:34.985066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:34.985608 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:34.985609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:34.985677 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:34.985680 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:34.985731 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:12:34.985773 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":129,"threads_started":1}
I20260812 21:12:34.986129 13693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c7242138824986931440d6e9fc0ae5" candidate_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae09d773cde4a389802f3ab2deb96f2" is_pre_election: true
I20260812 21:12:34.986202 14194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c7242138824986931440d6e9fc0ae5" candidate_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2ba21c83be40ba92aa04257677bf6b" is_pre_election: true
I20260812 21:12:34.986331 14194 raft_consensus.cc:2454] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c888c18b7b9e4b5198d74efa5f0fdd99 in term 0.
I20260812 21:12:34.986331 13693 raft_consensus.cc:2454] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c888c18b7b9e4b5198d74efa5f0fdd99 in term 0.
I20260812 21:12:34.986567 13819 leader_election.cc:302] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c888c18b7b9e4b5198d74efa5f0fdd99, cae09d773cde4a389802f3ab2deb96f2; no voters:
I20260812 21:12:34.986704 14309 raft_consensus.cc:2781] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:12:34.986768 14309 raft_consensus.cc:482] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:12:34.986784 14309 raft_consensus.cc:3037] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:34.987520 14309 raft_consensus.cc:504] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.987646 14309 leader_election.cc:288] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [CANDIDATE]: Term 1 election: Requested vote from peers cae09d773cde4a389802f3ab2deb96f2 (127.11.207.129:39381), ed2ba21c83be40ba92aa04257677bf6b (127.11.207.131:46437)
I20260812 21:12:34.987818 14194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c7242138824986931440d6e9fc0ae5" candidate_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2ba21c83be40ba92aa04257677bf6b"
I20260812 21:12:34.987865 13693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c7242138824986931440d6e9fc0ae5" candidate_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae09d773cde4a389802f3ab2deb96f2"
I20260812 21:12:34.987914 14194 raft_consensus.cc:3037] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:34.987932 13693 raft_consensus.cc:3037] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:34.988871 13693 raft_consensus.cc:2454] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c888c18b7b9e4b5198d74efa5f0fdd99 in term 1.
I20260812 21:12:34.988871 14194 raft_consensus.cc:2454] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c888c18b7b9e4b5198d74efa5f0fdd99 in term 1.
I20260812 21:12:34.989064 13820 leader_election.cc:302] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c888c18b7b9e4b5198d74efa5f0fdd99, ed2ba21c83be40ba92aa04257677bf6b; no voters:
I20260812 21:12:34.989145 14309 raft_consensus.cc:2781] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:34.989310 14309 raft_consensus.cc:686] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 1 LEADER]: Becoming Leader. State: Replica: c888c18b7b9e4b5198d74efa5f0fdd99, State: Running, Role: LEADER
I20260812 21:12:34.989432 14309 consensus_queue.cc:237] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } }
I20260812 21:12:34.990332 13410 catalog_manager.cc:5495] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 reported cstate change: term changed from 0 to 1, leader changed from <none> to c888c18b7b9e4b5198d74efa5f0fdd99 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c888c18b7b9e4b5198d74efa5f0fdd99" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 36109 } health_report { overall_health: HEALTHY } } }
I20260812 21:12:35.039386 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.037033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.11.207.130:36109)
0812 21:12:35.037092 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.037094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.037104 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:12:35.037155 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.037156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.037181 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.037286 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:12:35.037428 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.037918 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.038646 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.038852 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.038909 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.039063 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:12:35.039157 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:12:35.039256 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.039285 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.039296 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:12:35.041512 14194 raft_consensus.cc:1261] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 1 FOLLOWER]: Refusing update from remote peer c888c18b7b9e4b5198d74efa5f0fdd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:35.041508 13693 raft_consensus.cc:1261] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 1 FOLLOWER]: Refusing update from remote peer c888c18b7b9e4b5198d74efa5f0fdd99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:35.041889 14309 consensus_queue.cc:1035] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae09d773cde4a389802f3ab2deb96f2" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 39381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:35.042039 14318 consensus_queue.cc:1035] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed2ba21c83be40ba92aa04257677bf6b" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 46437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:35.044305 14321 mvcc.cc:205] Tried to move back new op lower bound from 7317787259047088128 to 7317787258837258240. Current Snapshot: MvccSnapshot[applied={T|T < 7317787259047088128}]
I20260812 21:12:35.044557 14324 mvcc.cc:205] Tried to move back new op lower bound from 7317787259047088128 to 7317787258837258240. Current Snapshot: MvccSnapshot[applied={T|T < 7317787259047088128}]
I20260812 21:12:35.044901 14323 mvcc.cc:205] Tried to move back new op lower bound from 7317787259047088128 to 7317787258837258240. Current Snapshot: MvccSnapshot[applied={T|T < 7317787259047088128}]
I20260812 21:12:35.047328 13412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:35.047417 13412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:35.048143 13412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b5a53d298b94955bb93b9f29d1b105b: Sending DeleteTablet for 3 replicas of tablet 90c7242138824986931440d6e9fc0ae5
I20260812 21:12:35.048542 13895 tablet_service.cc:1514] Processing DeleteTablet for tablet 90c7242138824986931440d6e9fc0ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:35 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:57554
I20260812 21:12:35.048578 13660 tablet_service.cc:1514] Processing DeleteTablet for tablet 90c7242138824986931440d6e9fc0ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:35 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:34920
I20260812 21:12:35.048816 14338 tablet_replica.cc:330] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: stopping tablet replica
I20260812 21:12:35.048815 14150 tablet_service.cc:1514] Processing DeleteTablet for tablet 90c7242138824986931440d6e9fc0ae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:35 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:51856
I20260812 21:12:35.048842 14336 tablet_replica.cc:330] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: stopping tablet replica
I20260812 21:12:35.048933 14338 raft_consensus.cc:2229] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:35.049057 14338 raft_consensus.cc:2258] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:35.049052 14336 raft_consensus.cc:2229] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:12:35.049333 14336 raft_consensus.cc:2258] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:35.049427 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13538
I20260812 21:12:35.049525 14338 ts_tablet_manager.cc:1905] T 90c7242138824986931440d6e9fc0ae5 P cae09d773cde4a389802f3ab2deb96f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:35.050536 14336 ts_tablet_manager.cc:1905] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:35.052238 14336 ts_tablet_manager.cc:1918] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:35.052296 14336 log.cc:1196] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/90c7242138824986931440d6e9fc0ae5
I20260812 21:12:35.052536 14336 ts_tablet_manager.cc:1939] T 90c7242138824986931440d6e9fc0ae5 P c888c18b7b9e4b5198d74efa5f0fdd99: Deleting consensus metadata
I20260812 21:12:35.053194 13394 catalog_manager.cc:4841] TS c888c18b7b9e4b5198d74efa5f0fdd99 (127.11.207.130:36109): tablet 90c7242138824986931440d6e9fc0ae5 (table test-table [id=2282a5e3b273468a94b74921e6098b69]) successfully deleted
I20260812 21:12:35.063347 14339 tablet_replica.cc:330] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: stopping tablet replica
I20260812 21:12:35.063536 14339 raft_consensus.cc:2229] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:35.063686 14339 raft_consensus.cc:2258] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:35.064409 14339 ts_tablet_manager.cc:1905] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:35.066406 14339 ts_tablet_manager.cc:1918] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:35.066493 14339 log.cc:1196] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/90c7242138824986931440d6e9fc0ae5
I20260812 21:12:35.066752 14339 ts_tablet_manager.cc:1939] T 90c7242138824986931440d6e9fc0ae5 P ed2ba21c83be40ba92aa04257677bf6b: Deleting consensus metadata
I20260812 21:12:35.067334 13394 catalog_manager.cc:4841] TS ed2ba21c83be40ba92aa04257677bf6b (127.11.207.131:46437): tablet 90c7242138824986931440d6e9fc0ae5 (table test-table [id=2282a5e3b273468a94b74921e6098b69]) successfully deleted
W20260812 21:12:35.071054 13391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:39381 (error 108)
I20260812 21:12:35.071626 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.071484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:39381 (local address 127.0.0.1:34936)
0812 21:12:35.071543 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.071590 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:39381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 21:12:35.071647 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13783
W20260812 21:12:35.080677 13391 catalog_manager.cc:4584] TS cae09d773cde4a389802f3ab2deb96f2 (127.11.207.129:39381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90c7242138824986931440d6e9fc0ae5: Network error: Client connection negotiation failed: client connection to 127.11.207.129:39381: connect: Connection refused (error 111)
I20260812 21:12:35.088475 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14036
I20260812 21:12:35.103369 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 13358
2026-08-12T21:12:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2398 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T21:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:35Z Disabled control of system clock
I20260812 21:12:35.135478 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37051
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:34441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:37051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:35.218703 14344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:35.218952 14344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:35.218981 14344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:35.220157 14344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:35.220202 14344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:35.220220 14344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:35.220237 14344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:35.220252 14344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:35.221642 14344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:37051
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:37051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:35.221903 14344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:35.222180 14344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:35.227361 14356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:35.227428 14353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:35.227566 14344 server_base.cc:1034] running on GCE node
W20260812 21:12:35.227378 14350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:35.227910 14344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:35.228204 14344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:35.229350 14344 hybrid_clock.cc:648] HybridClock initialized: now 1786569155229327 us; error 36 us; skew 500 ppm
I20260812 21:12:35.234529 14344 webserver.cc:460] Webserver started at http://127.11.207.190:33649/ using document root <none> and password file <none>
I20260812 21:12:35.234779 14344 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:35.234835 14344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:35.234977 14344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:35.236132 14344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "3b9c0118af5d4c36b616af300c1404aa"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "7e430ce1b87d8efc78c0ec5c9385292c"
server_key_iv: "8962253a7271e6b4bd8fc33fa91eab7b"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.236503 14344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "3b9c0118af5d4c36b616af300c1404aa"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "7e430ce1b87d8efc78c0ec5c9385292c"
server_key_iv: "8962253a7271e6b4bd8fc33fa91eab7b"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.237818 14344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:35.238515 14366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.238663 14344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:35.238806 14344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "3b9c0118af5d4c36b616af300c1404aa"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "7e430ce1b87d8efc78c0ec5c9385292c"
server_key_iv: "8962253a7271e6b4bd8fc33fa91eab7b"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.238871 14344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:35.253217 14344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:35.253552 14344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:35.253686 14344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:35.257525 14344 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:37051
I20260812 21:12:35.257563 14476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:37051 every 8 connection(s)
I20260812 21:12:35.257892 14344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:35.258903 14479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:35.260367 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14344
I20260812 21:12:35.260478 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:35.260671 12094 external_mini_cluster.cc:1428] Setting key 546926cb9257a4d652eac676b9af0306
I20260812 21:12:35.261502 14479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa: Bootstrap starting.
I20260812 21:12:35.262099 14479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:35.262391 14479 log.cc:824] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:35.263047 14479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa: No bootstrap required, opened a new log
I20260812 21:12:35.264865 14479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9c0118af5d4c36b616af300c1404aa" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37051 } }
I20260812 21:12:35.265040 14479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:35.265091 14479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9c0118af5d4c36b616af300c1404aa, State: Initialized, Role: FOLLOWER
I20260812 21:12:35.265223 14479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9c0118af5d4c36b616af300c1404aa" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37051 } }
I20260812 21:12:35.265296 14479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:35.265328 14479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:35.265379 14479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:35.265939 14479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9c0118af5d4c36b616af300c1404aa" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37051 } }
I20260812 21:12:35.265974 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.261212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.11.207.190:37051)
0812 21:12:35.261431 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.261439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.261464 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:12:35.261927 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.261935 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.262004 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.262144 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:35.262378 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.263302 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.264078 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.264533 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.264644 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.265325 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:35.265329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:35.265330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:35.265529 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:35.265563 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:35.265574 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:35.265627 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.265705 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.265790 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260812 21:12:35.266049 14479 leader_election.cc:302] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b9c0118af5d4c36b616af300c1404aa; no voters:
I20260812 21:12:35.266212 14479 leader_election.cc:288] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:35.266250 14489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:35.266386 14489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [term 1 LEADER]: Becoming Leader. State: Replica: 3b9c0118af5d4c36b616af300c1404aa, State: Running, Role: LEADER
I20260812 21:12:35.266482 14489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9c0118af5d4c36b616af300c1404aa" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37051 } }
I20260812 21:12:35.266783 14479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:35.266824 14489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b9c0118af5d4c36b616af300c1404aa. Latest consensus state: current_term: 1 leader_uuid: "3b9c0118af5d4c36b616af300c1404aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9c0118af5d4c36b616af300c1404aa" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37051 } } }
I20260812 21:12:35.266849 14491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b9c0118af5d4c36b616af300c1404aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b9c0118af5d4c36b616af300c1404aa" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 37051 } } }
I20260812 21:12:35.266913 14489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:35.266927 14491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:35.267589 14494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:35.268091 14494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:35.269575 14494 catalog_manager.cc:1347] Generated new cluster ID: 9939f1feba104cbebe51d47214bd739f
I20260812 21:12:35.269627 14494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:35.278698 14494 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:35.279270 14494 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:35.284827 14494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa: Generated new TSK 0
I20260812 21:12:35.285017 14494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:35.288882 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37051
--builtin_ntp_servers=127.11.207.148:34441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:35.374333 14525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:35.374548 14525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:35.374576 14525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:35.375777 14525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:35.375819 14525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:35.375854 14525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:35.377293 14525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:35.377532 14525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:35.377830 14525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:35.382803 14545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:35.382761 14541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:35.382815 14537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:35.383396 14525 server_base.cc:1034] running on GCE node
I20260812 21:12:35.383618 14525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:35.383867 14525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:35.385010 14525 hybrid_clock.cc:648] HybridClock initialized: now 1786569155384996 us; error 31 us; skew 500 ppm
I20260812 21:12:35.386198 14525 webserver.cc:460] Webserver started at http://127.11.207.129:39359/ using document root <none> and password file <none>
I20260812 21:12:35.386416 14525 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:35.386468 14525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:35.386595 14525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:35.387652 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "c8dec41b239b445d96426898e8edddea"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "113465ee7c1621d42f39cd6472ef6973"
server_key_iv: "089c06619565dbca3eff1baa9e8c100b"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.387990 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "c8dec41b239b445d96426898e8edddea"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "113465ee7c1621d42f39cd6472ef6973"
server_key_iv: "089c06619565dbca3eff1baa9e8c100b"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.389343 14525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:12:35.390031 14554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.390194 14525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:35.390290 14525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "c8dec41b239b445d96426898e8edddea"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "113465ee7c1621d42f39cd6472ef6973"
server_key_iv: "089c06619565dbca3eff1baa9e8c100b"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.390354 14525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:35.414510 14525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:35.414866 14525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:35.415009 14525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:35.415244 14525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:35.415576 14525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:35.415611 14525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.415642 14525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:35.415658 14525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.421294 14525 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:45343
I20260812 21:12:35.421370 14753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:45343 every 8 connection(s)
I20260812 21:12:35.421676 14525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:35.425347 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14525
I20260812 21:12:35.425439 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:12:35.425616 12094 external_mini_cluster.cc:1428] Setting key 3b1e4fc4563c0bfe0513e74e58c54359
I20260812 21:12:35.426553 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37051
--builtin_ntp_servers=127.11.207.148:34441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:12:35.427022 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.422401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:38123 (local address 127.11.207.190:37051)
0812 21:12:35.422467 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.422469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.422726 (+ 257us) server_negotiation.cc:407] Connection header received
0812 21:12:35.423255 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.423256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.423269 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.423329 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:35.424050 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.424592 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.425585 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.425830 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.425893 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.426783 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:35.426785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:35.426785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:35.426791 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:35.426808 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:35.426811 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:35.426857 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.426943 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.426966 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:12:35.427270 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.422383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37051 (local address 127.11.207.129:38123)
0812 21:12:35.422597 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.422632 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.423149 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.423400 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.423403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.423412 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:35.423955 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.423961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.424675 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.424676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.425503 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.425507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.425916 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.425918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.426065 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.426589 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:35.426590 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:35.426632 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:35.426662 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:35.426868 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:35.426875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:35.426878 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.426944 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.427051 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260812 21:12:35.427834 14755 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37051
I20260812 21:12:35.427932 14755 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:35.428124 14755 heartbeater.cc:511] Master 127.11.207.190:37051 requested a full tablet report, sending...
I20260812 21:12:35.428655 14401 ts_manager.cc:194] Registered new tserver with Master: c8dec41b239b445d96426898e8edddea (127.11.207.129:45343)
I20260812 21:12:35.429275 14401 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.129:38123
W20260812 21:12:35.512104 14759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:35.512336 14759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:35.512384 14759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:35.513547 14759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:35.513588 14759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:35.513612 14759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:35.514951 14759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:35.515156 14759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:35.515417 14759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:12:35.520588 14759 server_base.cc:1034] running on GCE node
W20260812 21:12:35.520511 14767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:35.520506 14771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:35.520506 14768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:35.520957 14759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:35.521211 14759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:35.522377 14759 hybrid_clock.cc:648] HybridClock initialized: now 1786569155522348 us; error 43 us; skew 500 ppm
I20260812 21:12:35.523617 14759 webserver.cc:460] Webserver started at http://127.11.207.130:45135/ using document root <none> and password file <none>
I20260812 21:12:35.523886 14759 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:35.523940 14759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:35.524065 14759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:35.525148 14759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "c99f628b821e431683d569f7cf44192a"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "51de74c5d0af87a1fc7d0bb931eb2b4d"
server_key_iv: "5d15bf0d0126aa8d9cbdbc9e1d2d9b61"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.525471 14759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "c99f628b821e431683d569f7cf44192a"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "51de74c5d0af87a1fc7d0bb931eb2b4d"
server_key_iv: "5d15bf0d0126aa8d9cbdbc9e1d2d9b61"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.526693 14759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:12:35.527453 14782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.527621 14759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:35.527699 14759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "c99f628b821e431683d569f7cf44192a"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "51de74c5d0af87a1fc7d0bb931eb2b4d"
server_key_iv: "5d15bf0d0126aa8d9cbdbc9e1d2d9b61"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.527772 14759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:35.540218 14759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:35.540555 14759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:35.540683 14759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:35.540902 14759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:35.541247 14759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:35.541289 14759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.541318 14759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:35.541342 14759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.547049 14759 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:46071
I20260812 21:12:35.547111 14990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:46071 every 8 connection(s)
I20260812 21:12:35.547421 14759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:12:35.552618 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.548130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:45781 (local address 127.11.207.190:37051)
0812 21:12:35.548186 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.548188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.548445 (+ 257us) server_negotiation.cc:407] Connection header received
0812 21:12:35.549013 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.549014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.549028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.549080 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:35.549766 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.550262 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.551258 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.551449 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.551504 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.552309 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:35.552311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:35.552311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:35.552316 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:35.552333 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:35.552337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:35.552421 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.552503 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.552533 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:12:35.552838 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.548092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37051 (local address 127.11.207.130:45781)
0812 21:12:35.548309 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.548367 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.548911 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.549152 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.549157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.549167 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:35.549671 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.549676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.550345 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.550346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.551171 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.551174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.551517 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.551518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.551654 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.552171 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:35.552172 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:35.552204 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:35.552228 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:35.552428 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:35.552435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:35.552438 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.552501 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.552610 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260812 21:12:35.552846 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14759
I20260812 21:12:35.552942 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:12:35.553165 12094 external_mini_cluster.cc:1428] Setting key 7bf45eeffa85ad8bd65721931bc10167
I20260812 21:12:35.553299 14992 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37051
I20260812 21:12:35.553401 14992 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:35.553594 14992 heartbeater.cc:511] Master 127.11.207.190:37051 requested a full tablet report, sending...
I20260812 21:12:35.553967 14401 ts_manager.cc:194] Registered new tserver with Master: c99f628b821e431683d569f7cf44192a (127.11.207.130:46071)
I20260812 21:12:35.554152 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37051
--builtin_ntp_servers=127.11.207.148:34441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:12:35.554407 14401 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.130:45781
W20260812 21:12:35.637916 14998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:35.638134 14998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:35.638165 14998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:35.639322 14998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:35.639361 14998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:35.639395 14998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:35.640779 14998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:34441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:37051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:35.641060 14998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:35.641345 14998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:35.646499 15006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:35.646603 14998 server_base.cc:1034] running on GCE node
W20260812 21:12:35.646507 15007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:35.646759 15011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:35.646994 14998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:35.647254 14998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:35.648414 14998 hybrid_clock.cc:648] HybridClock initialized: now 1786569155648395 us; error 37 us; skew 500 ppm
I20260812 21:12:35.649755 14998 webserver.cc:460] Webserver started at http://127.11.207.131:39797/ using document root <none> and password file <none>
I20260812 21:12:35.649979 14998 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:35.650032 14998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:35.650169 14998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:35.651262 14998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "d70c001060114d29ae19f17d9929dd4d"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "850297673079ca3ae10f22754a137abc"
server_key_iv: "fa99f9843b9c719584a706f47de788f3"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.651595 14998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "d70c001060114d29ae19f17d9929dd4d"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "850297673079ca3ae10f22754a137abc"
server_key_iv: "fa99f9843b9c719584a706f47de788f3"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.652961 14998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:35.653772 15019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.653985 14998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:35.654083 14998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "d70c001060114d29ae19f17d9929dd4d"
format_stamp: "Formatted at 2026-08-12 21:12:35 on dist-test-slave-lkx2"
server_key: "850297673079ca3ae10f22754a137abc"
server_key_iv: "fa99f9843b9c719584a706f47de788f3"
server_key_version: "encryptionkey@0"
I20260812 21:12:35.654146 14998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:35.675622 14998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:35.675976 14998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:35.676112 14998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:35.676391 14998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:35.676766 14998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:35.676802 14998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.676834 14998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:35.676857 14998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:35.682435 14998 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:44569
I20260812 21:12:35.682483 15223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:44569 every 8 connection(s)
I20260812 21:12:35.682827 14998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:35.687925 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.683542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:57579 (local address 127.11.207.190:37051)
0812 21:12:35.683596 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.683598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.683830 (+ 232us) server_negotiation.cc:407] Connection header received
0812 21:12:35.684397 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.684398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.684412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.684469 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:35.685180 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.685676 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.686638 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.686829 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.686886 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.687669 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:35.687670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:35.687671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:35.687676 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:35.687689 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:35.687693 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:35.687734 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.687816 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.687843 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:12:35.688171 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.683514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:37051 (local address 127.11.207.131:57579)
0812 21:12:35.683708 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.683744 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.684279 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.684552 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.684556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.684568 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:35.685085 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.685089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.685757 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.685758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.686551 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.686553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.686899 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.686900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.687043 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.687526 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:35.687527 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:35.687556 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:35.687579 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:35.687740 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:35.687746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:35.687749 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.687812 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.687925 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260812 21:12:35.688647 15227 heartbeater.cc:347] Connected to a master server at 127.11.207.190:37051
I20260812 21:12:35.688737 15227 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:35.688911 15227 heartbeater.cc:511] Master 127.11.207.190:37051 requested a full tablet report, sending...
I20260812 21:12:35.689281 14401 ts_manager.cc:194] Registered new tserver with Master: d70c001060114d29ae19f17d9929dd4d (127.11.207.131:44569)
I20260812 21:12:35.689615 14401 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.131:57579
I20260812 21:12:35.690099 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 14998
I20260812 21:12:35.690194 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:12:35.690387 12094 external_mini_cluster.cc:1428] Setting key af28bd4d1a53e010cb25085f60395096
I20260812 21:12:35.691256 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:12:35.695335 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.692659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.11.207.190:37051)
0812 21:12:35.692701 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.692703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.692818 (+ 115us) server_negotiation.cc:407] Connection header received
0812 21:12:35.692857 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.692858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.692867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.692911 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:35.693027 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.693563 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.694298 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.694471 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.694515 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.695097 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:35.695098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:35.695098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:35.695103 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:35.695115 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:35.695118 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:35.695186 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.695245 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.695255 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:12:35.696908 14401 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:12:35.698459 14401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:12:35.704730 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.700491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45343 (local address 127.0.0.1:38142)
0812 21:12:35.700945 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.700954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.701009 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.701316 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.701317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.701319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:35.701358 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.701360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.702564 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.702565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.703773 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.703775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.704439 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.704440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.704514 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.704527 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.704597 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.704649 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260812 21:12:35.704746 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.700575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.11.207.129:45343)
0812 21:12:35.700876 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.700880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.701083 (+ 203us) server_negotiation.cc:407] Connection header received
0812 21:12:35.701135 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.701137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.701153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.701219 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:35.701436 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.702495 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.703857 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.704374 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.704441 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.704535 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.704612 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.704650 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":75,"threads_started":1}
I20260812 21:12:35.705017 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.700488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44569 (local address 127.0.0.1:43740)
0812 21:12:35.700617 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.700659 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.700718 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.701000 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.701003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.701006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:35.701080 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.701081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.702424 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.702425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.703744 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.703747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.704796 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.704797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.704861 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.704865 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.704913 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.704949 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":32,"thread_start_us":78,"threads_started":1}
I20260812 21:12:35.705266 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.700681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.11.207.131:44569)
0812 21:12:35.700828 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.700833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.700844 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:12:35.700886 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.700889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.700905 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.700987 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:35.701216 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.702337 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.703840 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.704613 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.704683 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.704761 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.705127 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.705176 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260812 21:12:35.705458 14631 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0e518db57d4332ba9e491b80802c83 (DEFAULT_TABLE table=test-table [id=5bde9ef38dc543d38b7997c79c3cf910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:35.705852 14631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0e518db57d4332ba9e491b80802c83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:35.705924 15103 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0e518db57d4332ba9e491b80802c83 (DEFAULT_TABLE table=test-table [id=5bde9ef38dc543d38b7997c79c3cf910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:35.706240 15103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0e518db57d4332ba9e491b80802c83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:35.706368 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.700718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:46071 (local address 127.0.0.1:42078)
0812 21:12:35.702595 (+ 1877us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.702604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.702647 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.702826 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.702827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.702829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:35.702868 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.702869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.703851 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.703853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.705328 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.705331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.706141 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.706142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.706201 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.706205 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.706263 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.706300 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1859,"thread_start_us":76,"threads_started":1}
I20260812 21:12:35.706380 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.700931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.11.207.130:46071)
0812 21:12:35.701145 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.701150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.702677 (+ 1527us) server_negotiation.cc:407] Connection header received
0812 21:12:35.702726 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.702729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.702744 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.702818 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:35.702919 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.703765 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.705425 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.706061 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.706128 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.706213 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.706265 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.706301 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260812 21:12:35.709453 15256 tablet_bootstrap.cc:492] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: Bootstrap starting.
I20260812 21:12:35.709522 15254 tablet_bootstrap.cc:492] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Bootstrap starting.
I20260812 21:12:35.710186 15256 tablet_bootstrap.cc:654] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:35.710175 15254 tablet_bootstrap.cc:654] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:35.710134 14874 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0e518db57d4332ba9e491b80802c83 (DEFAULT_TABLE table=test-table [id=5bde9ef38dc543d38b7997c79c3cf910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:35.710453 14874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0e518db57d4332ba9e491b80802c83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:35.710538 15256 log.cc:824] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:35.710538 15254 log.cc:824] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:35.711300 15256 tablet_bootstrap.cc:492] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: No bootstrap required, opened a new log
I20260812 21:12:35.711380 15256 ts_tablet_manager.cc:1397] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:12:35.711542 15254 tablet_bootstrap.cc:492] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: No bootstrap required, opened a new log
I20260812 21:12:35.711608 15254 ts_tablet_manager.cc:1397] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:12:35.713376 15260 tablet_bootstrap.cc:492] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Bootstrap starting.
I20260812 21:12:35.713996 15260 tablet_bootstrap.cc:654] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:35.714278 15260 log.cc:824] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:35.714155 15254 raft_consensus.cc:348] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.714218 15256 raft_consensus.cc:348] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.714334 15254 raft_consensus.cc:374] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:35.714334 15256 raft_consensus.cc:374] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:35.714380 15254 raft_consensus.cc:729] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d70c001060114d29ae19f17d9929dd4d, State: Initialized, Role: FOLLOWER
I20260812 21:12:35.714385 15256 raft_consensus.cc:729] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8dec41b239b445d96426898e8edddea, State: Initialized, Role: FOLLOWER
I20260812 21:12:35.714527 15256 consensus_queue.cc:260] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.714527 15254 consensus_queue.cc:260] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.714811 15256 ts_tablet_manager.cc:1428] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 21:12:35.714811 15254 ts_tablet_manager.cc:1428] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:12:35.714857 15227 heartbeater.cc:503] Master 127.11.207.190:37051 was elected leader, sending a full tablet report...
I20260812 21:12:35.715090 14755 heartbeater.cc:503] Master 127.11.207.190:37051 was elected leader, sending a full tablet report...
I20260812 21:12:35.715404 15260 tablet_bootstrap.cc:492] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: No bootstrap required, opened a new log
I20260812 21:12:35.715488 15260 ts_tablet_manager.cc:1397] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:12:35.717440 15260 raft_consensus.cc:348] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.717572 15260 raft_consensus.cc:374] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:35.717593 15260 raft_consensus.cc:729] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c99f628b821e431683d569f7cf44192a, State: Initialized, Role: FOLLOWER
I20260812 21:12:35.717715 15260 consensus_queue.cc:260] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.717958 15260 ts_tablet_manager.cc:1428] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:12:35.718019 14992 heartbeater.cc:503] Master 127.11.207.190:37051 was elected leader, sending a full tablet report...
I20260812 21:12:35.784319 15268 raft_consensus.cc:482] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:35.784505 15268 raft_consensus.cc:504] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.784937 15268 leader_election.cc:288] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d70c001060114d29ae19f17d9929dd4d (127.11.207.131:44569), c8dec41b239b445d96426898e8edddea (127.11.207.129:45343)
I20260812 21:12:35.787843 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.785152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:44569 (local address 127.11.207.130:53741)
0812 21:12:35.785230 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.785242 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.785289 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.785452 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.785454 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.785456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:35.785530 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.785532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.786285 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.786286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.787087 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.787089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.787606 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.787607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.787680 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.787684 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.787728 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.787771 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 21:12:35.787843 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.785236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:53741 (local address 127.11.207.131:44569)
0812 21:12:35.785289 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.785291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.785304 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:12:35.785360 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.785360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.785371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.785443 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:35.785594 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.786214 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.787167 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.787534 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.787598 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.787606 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.787739 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.787777 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:12:35.788133 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.785328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:58375 (local address 127.11.207.129:45343)
0812 21:12:35.785366 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.785368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.785711 (+ 343us) server_negotiation.cc:407] Connection header received
0812 21:12:35.785745 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.785747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.785760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.785814 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:35.785931 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.786391 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.787336 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.787733 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.787807 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.787817 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.788012 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.788059 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:12:35.788185 15145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0e518db57d4332ba9e491b80802c83" candidate_uuid: "c99f628b821e431683d569f7cf44192a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d70c001060114d29ae19f17d9929dd4d" is_pre_election: true
I20260812 21:12:35.788368 15145 raft_consensus.cc:2454] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c99f628b821e431683d569f7cf44192a in term 0.
I20260812 21:12:35.788386 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.785300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45343 (local address 127.11.207.130:58375)
0812 21:12:35.785640 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.785649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:35.785697 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:35.785816 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:35.785817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:35.785818 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:35.785882 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.785884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.786478 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.786480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.787239 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:35.787241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.787828 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.787829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:35.787906 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.787910 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:35.788230 (+ 320us) client_negotiation.cc:240] Negotiation successful
0812 21:12:35.788282 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":118,"threads_started":1}
I20260812 21:12:35.788673 14798 leader_election.cc:302] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c99f628b821e431683d569f7cf44192a, d70c001060114d29ae19f17d9929dd4d; no voters:
I20260812 21:12:35.788740 14671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0e518db57d4332ba9e491b80802c83" candidate_uuid: "c99f628b821e431683d569f7cf44192a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8dec41b239b445d96426898e8edddea" is_pre_election: true
I20260812 21:12:35.788805 15268 raft_consensus.cc:2781] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:12:35.788869 15268 raft_consensus.cc:482] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:12:35.788897 15268 raft_consensus.cc:3037] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:35.788897 14671 raft_consensus.cc:2454] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c99f628b821e431683d569f7cf44192a in term 0.
I20260812 21:12:35.789659 15268 raft_consensus.cc:504] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.789800 15268 leader_election.cc:288] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [CANDIDATE]: Term 1 election: Requested vote from peers d70c001060114d29ae19f17d9929dd4d (127.11.207.131:44569), c8dec41b239b445d96426898e8edddea (127.11.207.129:45343)
I20260812 21:12:35.789956 14671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0e518db57d4332ba9e491b80802c83" candidate_uuid: "c99f628b821e431683d569f7cf44192a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8dec41b239b445d96426898e8edddea"
I20260812 21:12:35.789981 15145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0e518db57d4332ba9e491b80802c83" candidate_uuid: "c99f628b821e431683d569f7cf44192a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d70c001060114d29ae19f17d9929dd4d"
I20260812 21:12:35.790040 14671 raft_consensus.cc:3037] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:35.790061 15145 raft_consensus.cc:3037] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:35.790807 14671 raft_consensus.cc:2454] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c99f628b821e431683d569f7cf44192a in term 1.
I20260812 21:12:35.790830 15145 raft_consensus.cc:2454] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c99f628b821e431683d569f7cf44192a in term 1.
I20260812 21:12:35.790987 14789 leader_election.cc:302] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8dec41b239b445d96426898e8edddea, c99f628b821e431683d569f7cf44192a; no voters:
I20260812 21:12:35.791087 15268 raft_consensus.cc:2781] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:35.791281 15268 raft_consensus.cc:686] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 1 LEADER]: Becoming Leader. State: Replica: c99f628b821e431683d569f7cf44192a, State: Running, Role: LEADER
I20260812 21:12:35.791433 15268 consensus_queue.cc:237] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } }
I20260812 21:12:35.792448 14400 catalog_manager.cc:5495] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a reported cstate change: term changed from 0 to 1, leader changed from <none> to c99f628b821e431683d569f7cf44192a (127.11.207.130). New cstate: current_term: 1 leader_uuid: "c99f628b821e431683d569f7cf44192a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c99f628b821e431683d569f7cf44192a" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 46071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 } health_report { overall_health: UNKNOWN } } }
W20260812 21:12:35.797907 14995 tablet.cc:2369] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:12:35.820784 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.818319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.11.207.130:46071)
0812 21:12:35.818382 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:35.818384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:35.818392 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:12:35.818432 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:35.818433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:35.818455 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:35.818517 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:12:35.818701 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.819224 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.820015 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:35.820217 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:35.820281 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:35.820469 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:12:35.820564 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:12:35.820650 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:35.820670 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 21:12:35.820680 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:12:35.822944 15145 raft_consensus.cc:1261] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 1 FOLLOWER]: Refusing update from remote peer c99f628b821e431683d569f7cf44192a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:35.822944 14671 raft_consensus.cc:1261] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 1 FOLLOWER]: Refusing update from remote peer c99f628b821e431683d569f7cf44192a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:35.823360 15268 consensus_queue.cc:1035] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d70c001060114d29ae19f17d9929dd4d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 44569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:35.823510 15270 consensus_queue.cc:1035] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8dec41b239b445d96426898e8edddea" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:35.826159 15278 mvcc.cc:205] Tried to move back new op lower bound from 7317787262247411712 to 7317787262122450944. Current Snapshot: MvccSnapshot[applied={T|T < 7317787262247411712}]
I20260812 21:12:35.826472 15281 mvcc.cc:205] Tried to move back new op lower bound from 7317787262247411712 to 7317787262122450944. Current Snapshot: MvccSnapshot[applied={T|T < 7317787262247411712}]
I20260812 21:12:35.828397 14401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:35.828481 14401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:35.829211 14401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b9c0118af5d4c36b616af300c1404aa: Sending DeleteTablet for 3 replicas of tablet 6a0e518db57d4332ba9e491b80802c83
I20260812 21:12:35.829635 14874 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0e518db57d4332ba9e491b80802c83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:35 UTC) from {username='slave'} at 127.0.0.1:42078
I20260812 21:12:35.829665 14631 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0e518db57d4332ba9e491b80802c83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:35 UTC) from {username='slave'} at 127.0.0.1:38142
I20260812 21:12:35.829886 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14525
I20260812 21:12:35.829911 15301 tablet_replica.cc:330] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea: stopping tablet replica
I20260812 21:12:35.829921 15103 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0e518db57d4332ba9e491b80802c83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:35 UTC) from {username='slave'} at 127.0.0.1:43740
I20260812 21:12:35.830015 15301 raft_consensus.cc:2229] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:35.830114 15301 raft_consensus.cc:2258] T 6a0e518db57d4332ba9e491b80802c83 P c8dec41b239b445d96426898e8edddea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:35.830272 15302 tablet_replica.cc:330] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: stopping tablet replica
I20260812 21:12:35.830410 15302 raft_consensus.cc:2229] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:35.830610 15302 raft_consensus.cc:2258] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:35.830891 15296 tablet_replica.cc:330] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: stopping tablet replica
I20260812 21:12:35.831069 15296 raft_consensus.cc:2229] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:12:35.831239 15296 raft_consensus.cc:2258] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:35.833755 15296 ts_tablet_manager.cc:1905] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:35.833917 15302 ts_tablet_manager.cc:1905] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:35.835770 15302 ts_tablet_manager.cc:1918] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:35.835856 15302 log.cc:1196] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/6a0e518db57d4332ba9e491b80802c83
I20260812 21:12:35.836112 15302 ts_tablet_manager.cc:1939] T 6a0e518db57d4332ba9e491b80802c83 P d70c001060114d29ae19f17d9929dd4d: Deleting consensus metadata
I20260812 21:12:35.835811 15296 ts_tablet_manager.cc:1918] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:35.836333 15296 log.cc:1196] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/6a0e518db57d4332ba9e491b80802c83
I20260812 21:12:35.836666 14375 catalog_manager.cc:4841] TS d70c001060114d29ae19f17d9929dd4d (127.11.207.131:44569): tablet 6a0e518db57d4332ba9e491b80802c83 (table test-table [id=5bde9ef38dc543d38b7997c79c3cf910]) successfully deleted
I20260812 21:12:35.837086 15296 ts_tablet_manager.cc:1939] T 6a0e518db57d4332ba9e491b80802c83 P c99f628b821e431683d569f7cf44192a: Deleting consensus metadata
I20260812 21:12:35.837651 14375 catalog_manager.cc:4841] TS c99f628b821e431683d569f7cf44192a (127.11.207.130:46071): tablet 6a0e518db57d4332ba9e491b80802c83 (table test-table [id=5bde9ef38dc543d38b7997c79c3cf910]) successfully deleted
W20260812 21:12:35.848191 14371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:45343 (error 108)
I20260812 21:12:35.848286 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14759
I20260812 21:12:35.851717 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:35.851286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45343 (local address 127.0.0.1:38152)
0812 21:12:35.851609 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:35.851670 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296}
W20260812 21:12:35.852213 14371 catalog_manager.cc:4584] TS c8dec41b239b445d96426898e8edddea (127.11.207.129:45343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a0e518db57d4332ba9e491b80802c83: Network error: Client connection negotiation failed: client connection to 127.11.207.129:45343: connect: Connection refused (error 111)
I20260812 21:12:35.857218 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14998
I20260812 21:12:35.872071 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 14344
2026-08-12T21:12:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (772 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:35 dist-test-slave-lkx2 krb5kdc[15307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:35 dist-test-slave-lkx2 krb5kdc[15307](info): set up 2 sockets
Aug 12 21:12:35 dist-test-slave-lkx2 krb5kdc[15307](info): commencing operation
krb5kdc: starting...
W20260812 21:12:37.073434 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:12:37.198730 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41961
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:37.282770 15330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:37.282976 15330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:37.282997 15330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:37.284126 15330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:37.284160 15330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:37.284174 15330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:37.284185 15330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:37.284206 15330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:37.285646 15330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:37.285843 15330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:37.286031 15330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:37.290973 15343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.291005 15341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:37.291067 15330 server_base.cc:1034] running on GCE node
W20260812 21:12:37.291004 15345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:37.291455 15330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:37.291795 15330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:37.292961 15330 hybrid_clock.cc:648] HybridClock initialized: now 1786569157292945 us; error 27 us; skew 500 ppm
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:37.295768 15330 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:12:37.296200 15330 webserver.cc:460] Webserver started at http://127.11.207.190:36451/ using document root <none> and password file <none>
I20260812 21:12:37.296465 15330 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:37.296519 15330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:37.296645 15330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:37.297719 15330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "2159bb907c83463d92928e4fbe327be5"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "e2cd8b7537ceffb47058c153616c93d5"
server_key_iv: "618926dc7c3deb600cfec6e9e8f12694"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.298058 15330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "2159bb907c83463d92928e4fbe327be5"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "e2cd8b7537ceffb47058c153616c93d5"
server_key_iv: "618926dc7c3deb600cfec6e9e8f12694"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.299418 15330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:37.300174 15354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.300366 15330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:37.300468 15330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "2159bb907c83463d92928e4fbe327be5"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "e2cd8b7537ceffb47058c153616c93d5"
server_key_iv: "618926dc7c3deb600cfec6e9e8f12694"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.300542 15330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:37.308389 15330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:37.310425 15330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:37.310555 15330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:37.314478 15330 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41961
I20260812 21:12:37.314520 15439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41961 every 8 connection(s)
I20260812 21:12:37.314853 15330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:37.315724 15443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:37.318025 15443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5: Bootstrap starting.
I20260812 21:12:37.318598 15443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:37.318850 15443 log.cc:824] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:37.319554 15443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5: No bootstrap required, opened a new log
I20260812 21:12:37.321209 15443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2159bb907c83463d92928e4fbe327be5" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41961 } }
I20260812 21:12:37.321348 15443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:37.321369 15443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2159bb907c83463d92928e4fbe327be5, State: Initialized, Role: FOLLOWER
I20260812 21:12:37.321498 15443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2159bb907c83463d92928e4fbe327be5" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41961 } }
I20260812 21:12:37.321561 15443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:37.321580 15443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:37.321609 15443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:37.322122 15443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2159bb907c83463d92928e4fbe327be5" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41961 } }
I20260812 21:12:37.322216 15443 leader_election.cc:302] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2159bb907c83463d92928e4fbe327be5; no voters:
I20260812 21:12:37.322363 15443 leader_election.cc:288] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:37.322407 15449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:37.322566 15449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [term 1 LEADER]: Becoming Leader. State: Replica: 2159bb907c83463d92928e4fbe327be5, State: Running, Role: LEADER
I20260812 21:12:37.322616 15443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:37.322654 15449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2159bb907c83463d92928e4fbe327be5" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41961 } }
I20260812 21:12:37.323062 15453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2159bb907c83463d92928e4fbe327be5. Latest consensus state: current_term: 1 leader_uuid: "2159bb907c83463d92928e4fbe327be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2159bb907c83463d92928e4fbe327be5" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41961 } } }
I20260812 21:12:37.323149 15453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:37.323248 15451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2159bb907c83463d92928e4fbe327be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2159bb907c83463d92928e4fbe327be5" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41961 } } }
I20260812 21:12:37.323339 15451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:37.323380 15466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:37.323942 15466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:37.324460 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15330
I20260812 21:12:37.324574 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:37.324779 12094 external_mini_cluster.cc:1428] Setting key c8e7a15f1de4d59e5a72eb794b46b9ff
I20260812 21:12:37.325906 15466 catalog_manager.cc:1347] Generated new cluster ID: 5e8383c31c3d46368b088389a422b5bb
I20260812 21:12:37.325956 15466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:37.336565 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.325248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.11.207.190:41961)
0812 21:12:37.325949 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.325958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.325996 (+ 38us) server_negotiation.cc:407] Connection header received
0812 21:12:37.326111 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.326117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.326172 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.326310 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:37.327055 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.328546 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.329570 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.330202 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.330308 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.333046 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:37.333051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:37.333051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:37.333064 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:37.334953 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.335337 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.335340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.335340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.335440 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.335615 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.335615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.335615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.335933 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:37.336059 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.336163 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.336296 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":53,"threads_started":1}
I20260812 21:12:37.344962 15466 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:37.345496 15466 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:37.348591 15466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5: Generated new TSK 0
I20260812 21:12:37.348815 15466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.207.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
Entry for principal kudu/127.11.207.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab.
I20260812 21:12:37.384263 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41961
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:37.482966 15494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:37.483182 15494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:37.483206 15494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:37.484546 15494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:37.484606 15494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:37.484634 15494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:37.486142 15494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.129.keytab
--principal=kudu/127.11.207.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:37.486338 15494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:37.486529 15494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:37.491564 15501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.491564 15506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.491564 15504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:37.491882 15494 server_base.cc:1034] running on GCE node
I20260812 21:12:37.492094 15494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:37.492321 15494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:37.493496 15494 hybrid_clock.cc:648] HybridClock initialized: now 1786569157493484 us; error 31 us; skew 500 ppm
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:37.496031 15494 init.cc:377] Logged in from keytab as kudu/127.11.207.129@KRBTEST.COM (short username kudu)
I20260812 21:12:37.496521 15494 webserver.cc:460] Webserver started at http://127.11.207.129:42759/ using document root <none> and password file <none>
I20260812 21:12:37.496762 15494 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:37.496812 15494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:37.496935 15494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:37.498001 15494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "d8caef4dbb79421ea75beacff6c7e4f4"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "f36d5ecb3d4f8806017fe2e386755f97"
server_key_iv: "cee1fca1617aee6b3be62cee3100a057"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.498381 15494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "d8caef4dbb79421ea75beacff6c7e4f4"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "f36d5ecb3d4f8806017fe2e386755f97"
server_key_iv: "cee1fca1617aee6b3be62cee3100a057"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.499698 15494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:37.500429 15515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.500571 15494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:37.500648 15494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "d8caef4dbb79421ea75beacff6c7e4f4"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "f36d5ecb3d4f8806017fe2e386755f97"
server_key_iv: "cee1fca1617aee6b3be62cee3100a057"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.500725 15494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:37.519331 15494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:37.521442 15494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:37.521581 15494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:37.521816 15494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:37.522188 15494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:37.522225 15494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.522254 15494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:37.522279 15494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.527966 15494 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:45715
I20260812 21:12:37.528050 15728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:45715 every 8 connection(s)
I20260812 21:12:37.528365 15494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:37.531373 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15494
I20260812 21:12:37.531499 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:12:37.531733 12094 external_mini_cluster.cc:1428] Setting key d94774e11765a22c2b55c8c9ac5f75bd
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.129@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:37.538653 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.529103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:55289 (local address 127.11.207.190:41961)
0812 21:12:37.529178 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.529180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.529352 (+ 172us) server_negotiation.cc:407] Connection header received
0812 21:12:37.529861 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.529862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.529875 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.529921 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:37.530874 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.531573 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.532903 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.533174 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.533261 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.535631 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:37.535633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:37.535633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:37.535641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:37.537192 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.537615 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.537616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.537617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.537682 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.537953 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.537954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.537954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.538104 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:37.538189 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.538462 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.538556 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:12:37.538846 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.529061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41961 (local address 127.11.207.129:55289)
0812 21:12:37.529234 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:37.529274 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:37.529783 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:37.529987 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:37.529990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:37.530561 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:37.530766 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.530770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.531707 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.531708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.532818 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.532821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.533281 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.533282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.533407 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.533894 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:37.533903 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:37.535522 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:37.537285 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:37.537293 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:37.537294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:37.537538 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:37.537766 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:37.537767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:37.537767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:37.537887 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:37.538193 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:37.538200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:37.538328 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 21:12:37.538422 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 21:12:37.538552 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260812 21:12:37.539386 15729 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41961
I20260812 21:12:37.539490 15729 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:37.539677 15729 heartbeater.cc:511] Master 127.11.207.190:41961 requested a full tablet report, sending...
I20260812 21:12:37.540300 15382 ts_manager.cc:194] Registered new tserver with Master: d8caef4dbb79421ea75beacff6c7e4f4 (127.11.207.129:45715)
I20260812 21:12:37.541270 15382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.129@KRBTEST.COM'} at 127.11.207.129:55289
WARNING: no policy specified for kudu/127.11.207.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
Entry for principal kudu/127.11.207.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab.
I20260812 21:12:37.560835 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41961
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:37.643572 15737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:37.643823 15737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:37.643862 15737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:37.645056 15737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:37.645099 15737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:37.645134 15737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:37.646591 15737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.130.keytab
--principal=kudu/127.11.207.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:37.646852 15737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:37.647065 15737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:12:37.652071 15737 server_base.cc:1034] running on GCE node
W20260812 21:12:37.652072 15747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.652227 15750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.652091 15746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:37.652485 15737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:37.652765 15737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:37.653939 15737 hybrid_clock.cc:648] HybridClock initialized: now 1786569157653919 us; error 29 us; skew 500 ppm
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:37.656399 15737 init.cc:377] Logged in from keytab as kudu/127.11.207.130@KRBTEST.COM (short username kudu)
I20260812 21:12:37.656880 15737 webserver.cc:460] Webserver started at http://127.11.207.130:33675/ using document root <none> and password file <none>
I20260812 21:12:37.657109 15737 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:37.657158 15737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:37.657284 15737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:37.658321 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "84c51e48bc3c46268daa27d0cf9785d0"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "444db38168a357afb010548fd66d3a8d"
server_key_iv: "42e3c3d644388bb06a584e3d978e5eae"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.658665 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "84c51e48bc3c46268daa27d0cf9785d0"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "444db38168a357afb010548fd66d3a8d"
server_key_iv: "42e3c3d644388bb06a584e3d978e5eae"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.659974 15737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:12:37.660751 15763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.660928 15737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.661024 15737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "84c51e48bc3c46268daa27d0cf9785d0"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "444db38168a357afb010548fd66d3a8d"
server_key_iv: "42e3c3d644388bb06a584e3d978e5eae"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.661088 15737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:37.676463 15737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:37.679112 15737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:37.679255 15737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:37.679488 15737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:37.679821 15737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:37.679855 15737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.679885 15737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:37.679909 15737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.685659 15737 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:45391
I20260812 21:12:37.685727 15958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:45391 every 8 connection(s)
I20260812 21:12:37.686021 15737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:12:37.686678 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15737
I20260812 21:12:37.686805 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:12:37.687036 12094 external_mini_cluster.cc:1428] Setting key 6e6799ab42897d859a3a7ea5fc4710a7
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.130@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.131@KRBTEST.COM; defaulting to no policy
I20260812 21:12:37.696041 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.686706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:56973 (local address 127.11.207.190:41961)
0812 21:12:37.686760 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.686762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.687027 (+ 265us) server_negotiation.cc:407] Connection header received
0812 21:12:37.687594 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.687595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.687606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.687658 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:37.688722 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.689401 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.690894 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.691086 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.691143 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.693494 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:37.693496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:37.693496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:37.693504 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:37.694755 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.695132 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.695133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.695133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.695189 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.695427 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.695428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.695429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.695537 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:37.695593 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.695862 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.695951 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:12:37.696205 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.686711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41961 (local address 127.11.207.130:56973)
0812 21:12:37.686894 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:37.686941 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:37.687488 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:37.687731 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:37.687736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:37.688432 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:37.688636 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.688641 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.689536 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.689538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.690808 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.690812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.691170 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.691171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.691318 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.691802 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:37.691820 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:37.693401 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:37.694831 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:37.694840 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:37.694842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:37.695064 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:37.695253 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:37.695254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:37.695254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:37.695369 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:37.695593 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:37.695601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:37.695744 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 21:12:37.695833 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:12:37.695990 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260812 21:12:37.696736 15960 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41961
I20260812 21:12:37.696830 15960 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:37.696985 15960 heartbeater.cc:511] Master 127.11.207.190:41961 requested a full tablet report, sending...
I20260812 21:12:37.697382 15382 ts_manager.cc:194] Registered new tserver with Master: 84c51e48bc3c46268daa27d0cf9785d0 (127.11.207.130:45391)
I20260812 21:12:37.697736 15382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.130@KRBTEST.COM'} at 127.11.207.130:56973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
Entry for principal kudu/127.11.207.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab.
I20260812 21:12:37.714854 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41961
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:37.800622 15967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:37.800832 15967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:37.800865 15967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:37.802054 15967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:37.802096 15967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:37.802130 15967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:37.803642 15967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.131.keytab
--principal=kudu/127.11.207.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:37.803894 15967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:37.804126 15967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:37.809181 15975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.809204 15977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:37.809183 15974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:37.809414 15967 server_base.cc:1034] running on GCE node
I20260812 21:12:37.809608 15967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:37.809852 15967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:37.811005 15967 hybrid_clock.cc:648] HybridClock initialized: now 1786569157810994 us; error 25 us; skew 500 ppm
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:37.813625 15967 init.cc:377] Logged in from keytab as kudu/127.11.207.131@KRBTEST.COM (short username kudu)
I20260812 21:12:37.814090 15967 webserver.cc:460] Webserver started at http://127.11.207.131:34603/ using document root <none> and password file <none>
I20260812 21:12:37.814304 15967 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:37.814358 15967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:37.814489 15967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:37.815531 15967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "4fe47d9d14b4a304fab20bcf458455bd"
server_key_iv: "06b33cefa7178ecd3d68eabbce385127"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.815862 15967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "4fe47d9d14b4a304fab20bcf458455bd"
server_key_iv: "06b33cefa7178ecd3d68eabbce385127"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.817209 15967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:12:37.817986 15992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.818168 15967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:12:37.818267 15967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d"
format_stamp: "Formatted at 2026-08-12 21:12:37 on dist-test-slave-lkx2"
server_key: "4fe47d9d14b4a304fab20bcf458455bd"
server_key_iv: "06b33cefa7178ecd3d68eabbce385127"
server_key_version: "encryptionkey@0"
I20260812 21:12:37.818329 15967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:37.838001 15967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:37.840013 15967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:37.840163 15967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:37.840461 15967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:37.840787 15967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:37.840822 15967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.840852 15967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:37.840869 15967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:37.846592 15967 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:39353
I20260812 21:12:37.846648 16172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:39353 every 8 connection(s)
I20260812 21:12:37.846961 15967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:37.850646 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 15967
I20260812 21:12:37.850769 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:12:37.850943 12094 external_mini_cluster.cc:1428] Setting key 65ce57b73e9e892ed09821e56fae7f97
Aug 12 21:12:37 dist-test-slave-lkx2 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569157, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.131@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:37.856316 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.847682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:45535 (local address 127.11.207.190:41961)
0812 21:12:37.847742 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.847744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.847943 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:12:37.848485 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.848487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.848501 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.848537 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:37.849717 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.850213 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.851324 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.851608 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.851664 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.853753 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:37.853754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:37.853755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:37.853764 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:37.855024 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.855426 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.855428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.855428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.855486 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.855702 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.855703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.855704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.855825 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:37.855882 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.856146 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.856225 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:12:37.856525 16175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.847654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:41961 (local address 127.11.207.131:45535)
0812 21:12:37.847821 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:37.847864 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:37.848377 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:37.848832 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:37.848835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:37.849423 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:37.849620 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.849625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.850297 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.850298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.851251 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.851254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.851682 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.851683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.851819 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.852239 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:12:37.852248 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:37.853655 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:37.855115 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:37.855123 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:37.855125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:37.855359 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:37.855540 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:12:37.855541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:12:37.855541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:12:37.855647 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:12:37.855881 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:37.855888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:37.856023 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:12:37.856116 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 21:12:37.856285 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260812 21:12:37.856963 16173 heartbeater.cc:347] Connected to a master server at 127.11.207.190:41961
I20260812 21:12:37.857061 16173 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:37.857256 16173 heartbeater.cc:511] Master 127.11.207.190:41961 requested a full tablet report, sending...
I20260812 21:12:37.857636 15382 ts_manager.cc:194] Registered new tserver with Master: dd9e99e286cc49f9a9e5ccfab5d0820d (127.11.207.131:39353)
I20260812 21:12:37.858013 15382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.207.131@KRBTEST.COM'} at 127.11.207.131:45535
I20260812 21:12:37.862651 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:12:37.869576 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.863942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.11.207.190:41961)
0812 21:12:37.864000 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.864002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.864076 (+ 74us) server_negotiation.cc:407] Connection header received
0812 21:12:37.864144 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.864145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.864154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.864202 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:37.864787 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.865267 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.865957 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.866149 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.866203 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.867316 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:37.867317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:37.867318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:37.867325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:37.868544 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.868861 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.868862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.868862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.868917 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:37.869074 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:37.869074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:37.869075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:37.869224 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:37.869310 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.869422 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.869492 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:12:37.870957 15382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:12:37.872613 15382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:12:37.878930 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.874769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:39353 (local address 127.0.0.1:55110)
0812 21:12:37.875126 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:37.875135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:37.875181 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:37.875546 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:37.875547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:37.875549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:37.875583 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.875584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.876900 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.876901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.877944 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.877947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.878600 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.878680 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.878695 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:12:37.878763 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 21:12:37.878819 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":96,"threads_started":1}
I20260812 21:12:37.879016 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.874765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:45391 (local address 127.0.0.1:54340)
0812 21:12:37.874922 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:37.874963 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:37.875029 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:37.875594 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:37.875595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:37.875597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:37.875651 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.875652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.876949 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.876950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.877821 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.877824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.878727 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.878793 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.878797 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:37.878826 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 21:12:37.878869 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":1,"thread_start_us":48,"threads_started":1}
I20260812 21:12:37.878988 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.875037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.11.207.129:45715)
0812 21:12:37.875229 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.875234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.875243 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:12:37.875296 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.875297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.875311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.875382 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:37.875733 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.876540 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.877971 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.878669 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878739 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.878797 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.878862 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.878907 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260812 21:12:37.879143 16192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.874765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45715 (local address 127.0.0.1:58680)
0812 21:12:37.875009 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:37.875019 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:37.875072 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:37.875413 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:37.875417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:37.875419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:37.875491 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.875492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.876637 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.876638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.877884 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:37.877887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.878736 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:37.878796 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.878801 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:12:37.878847 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:12:37.878884 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260812 21:12:37.879294 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.875018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.11.207.130:45391)
0812 21:12:37.875430 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.875434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.875442 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:12:37.875486 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.875488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.875500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.875738 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:37.875815 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.876870 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.877902 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.878607 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878980 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.879045 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.879072 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.879121 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":91,"threads_started":1}
I20260812 21:12:37.879673 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:37.875219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.11.207.131:39353)
0812 21:12:37.875381 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:37.875385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:37.875395 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:12:37.875434 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:37.875436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:37.875452 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:37.875541 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:37.875646 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.876793 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878031 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:37.878517 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:37.878587 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:37.878650 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:37.879535 (+ 885us) server_negotiation.cc:299] Negotiation successful
0812 21:12:37.879588 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260812 21:12:37.879931 15851 tablet_service.cc:1467] Processing CreateTablet for tablet e4881e649fba4b529ce35d3e8c9ad361 (DEFAULT_TABLE table=test-table [id=fe493205464740acb6d88b608cfc7121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:37.879977 15600 tablet_service.cc:1467] Processing CreateTablet for tablet e4881e649fba4b529ce35d3e8c9ad361 (DEFAULT_TABLE table=test-table [id=fe493205464740acb6d88b608cfc7121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:37.880335 15600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4881e649fba4b529ce35d3e8c9ad361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:37.880328 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4881e649fba4b529ce35d3e8c9ad361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:37.880275 16060 tablet_service.cc:1467] Processing CreateTablet for tablet e4881e649fba4b529ce35d3e8c9ad361 (DEFAULT_TABLE table=test-table [id=fe493205464740acb6d88b608cfc7121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:37.880620 16060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4881e649fba4b529ce35d3e8c9ad361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:37.883093 16202 tablet_bootstrap.cc:492] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Bootstrap starting.
I20260812 21:12:37.883720 16203 tablet_bootstrap.cc:492] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Bootstrap starting.
I20260812 21:12:37.883790 16202 tablet_bootstrap.cc:654] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:37.884430 16202 log.cc:824] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:37.884488 16203 tablet_bootstrap.cc:654] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:37.884591 16205 tablet_bootstrap.cc:492] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: Bootstrap starting.
I20260812 21:12:37.885031 16203 log.cc:824] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:37.885310 16202 tablet_bootstrap.cc:492] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: No bootstrap required, opened a new log
I20260812 21:12:37.885394 16202 ts_tablet_manager.cc:1397] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:12:37.885376 16205 tablet_bootstrap.cc:654] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:37.885653 16203 tablet_bootstrap.cc:492] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: No bootstrap required, opened a new log
I20260812 21:12:37.885721 16203 ts_tablet_manager.cc:1397] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:12:37.886078 16205 log.cc:824] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:37.886871 16205 tablet_bootstrap.cc:492] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: No bootstrap required, opened a new log
I20260812 21:12:37.886943 16205 ts_tablet_manager.cc:1397] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:12:37.887355 16202 raft_consensus.cc:348] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:37.887435 16203 raft_consensus.cc:348] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:37.887537 16202 raft_consensus.cc:374] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:37.887563 16203 raft_consensus.cc:374] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:37.887570 16202 raft_consensus.cc:729] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c51e48bc3c46268daa27d0cf9785d0, State: Initialized, Role: FOLLOWER
I20260812 21:12:37.887607 16203 raft_consensus.cc:729] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9e99e286cc49f9a9e5ccfab5d0820d, State: Initialized, Role: FOLLOWER
I20260812 21:12:37.887686 16202 consensus_queue.cc:260] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:37.887741 16203 consensus_queue.cc:260] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:37.887959 16202 ts_tablet_manager.cc:1428] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:12:37.888010 16203 ts_tablet_manager.cc:1428] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:12:37.888049 16173 heartbeater.cc:503] Master 127.11.207.190:41961 was elected leader, sending a full tablet report...
I20260812 21:12:37.888106 15960 heartbeater.cc:503] Master 127.11.207.190:41961 was elected leader, sending a full tablet report...
I20260812 21:12:37.889482 16205 raft_consensus.cc:348] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:37.889606 16205 raft_consensus.cc:374] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:37.889626 16205 raft_consensus.cc:729] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8caef4dbb79421ea75beacff6c7e4f4, State: Initialized, Role: FOLLOWER
I20260812 21:12:37.889734 16205 consensus_queue.cc:260] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:37.889973 16205 ts_tablet_manager.cc:1428] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:12:37.890041 15729 heartbeater.cc:503] Master 127.11.207.190:41961 was elected leader, sending a full tablet report...
W20260812 21:12:37.936590 15961 tablet.cc:2369] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:12:38.029234 15732 tablet.cc:2369] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:12:38.097509 16174 tablet.cc:2369] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:12:38.115955 16216 raft_consensus.cc:482] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:38.116098 16216 raft_consensus.cc:504] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:38.116469 16216 leader_election.cc:288] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd9e99e286cc49f9a9e5ccfab5d0820d (127.11.207.131:39353), d8caef4dbb79421ea75beacff6c7e4f4 (127.11.207.129:45715)
I20260812 21:12:38.120482 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.116793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:41511 (local address 127.11.207.131:39353)
0812 21:12:38.116845 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.116847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.116858 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:12:38.116891 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.116892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.116903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.116963 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.117136 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.117903 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.119459 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.120043 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.120123 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.120131 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.120362 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.120407 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:12:38.120482 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.116816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:50379 (local address 127.11.207.129:45715)
0812 21:12:38.116872 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.116874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.117002 (+ 128us) server_negotiation.cc:407] Connection header received
0812 21:12:38.117048 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.117049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.117058 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.117093 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.117230 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.117934 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.119459 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.120043 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.120123 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.120131 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.120362 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.120405 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:12:38.120510 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.116672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:39353 (local address 127.11.207.130:41511)
0812 21:12:38.116735 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.116747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.116798 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.116989 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.116990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.116992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.117067 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.117069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.118004 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.118005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.119350 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.119354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.120150 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.120152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.120263 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.120266 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.120330 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.120407 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":106}
I20260812 21:12:38.120515 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.116715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45715 (local address 127.11.207.130:50379)
0812 21:12:38.116927 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.116934 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.116990 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.117109 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.117109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.117111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.117172 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.117174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.118004 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.118005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.119350 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.119354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.120150 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.120152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.120256 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.120261 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.120330 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.120408 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":13,"thread_start_us":132,"threads_started":1}
I20260812 21:12:38.120913 16095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4881e649fba4b529ce35d3e8c9ad361" candidate_uuid: "84c51e48bc3c46268daa27d0cf9785d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" is_pre_election: true
I20260812 21:12:38.121032 15642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4881e649fba4b529ce35d3e8c9ad361" candidate_uuid: "84c51e48bc3c46268daa27d0cf9785d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" is_pre_election: true
I20260812 21:12:38.121109 16095 raft_consensus.cc:2454] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c51e48bc3c46268daa27d0cf9785d0 in term 0.
I20260812 21:12:38.121182 15642 raft_consensus.cc:2454] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84c51e48bc3c46268daa27d0cf9785d0 in term 0.
I20260812 21:12:38.121392 15766 leader_election.cc:302] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c51e48bc3c46268daa27d0cf9785d0, dd9e99e286cc49f9a9e5ccfab5d0820d; no voters:
I20260812 21:12:38.121587 16216 raft_consensus.cc:2781] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:12:38.121697 16216 raft_consensus.cc:482] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:12:38.121728 16216 raft_consensus.cc:3037] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.122537 16216 raft_consensus.cc:504] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:38.122684 16216 leader_election.cc:288] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [CANDIDATE]: Term 1 election: Requested vote from peers dd9e99e286cc49f9a9e5ccfab5d0820d (127.11.207.131:39353), d8caef4dbb79421ea75beacff6c7e4f4 (127.11.207.129:45715)
I20260812 21:12:38.122871 15642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4881e649fba4b529ce35d3e8c9ad361" candidate_uuid: "84c51e48bc3c46268daa27d0cf9785d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8caef4dbb79421ea75beacff6c7e4f4"
I20260812 21:12:38.122910 16095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4881e649fba4b529ce35d3e8c9ad361" candidate_uuid: "84c51e48bc3c46268daa27d0cf9785d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d"
I20260812 21:12:38.122969 15642 raft_consensus.cc:3037] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.122985 16095 raft_consensus.cc:3037] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.123800 15642 raft_consensus.cc:2454] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c51e48bc3c46268daa27d0cf9785d0 in term 1.
I20260812 21:12:38.123812 16095 raft_consensus.cc:2454] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84c51e48bc3c46268daa27d0cf9785d0 in term 1.
I20260812 21:12:38.123956 15766 leader_election.cc:302] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84c51e48bc3c46268daa27d0cf9785d0, dd9e99e286cc49f9a9e5ccfab5d0820d; no voters:
I20260812 21:12:38.124069 16216 raft_consensus.cc:2781] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:38.124243 16216 raft_consensus.cc:686] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 1 LEADER]: Becoming Leader. State: Replica: 84c51e48bc3c46268daa27d0cf9785d0, State: Running, Role: LEADER
I20260812 21:12:38.124405 16216 consensus_queue.cc:237] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } }
I20260812 21:12:38.125253 15381 catalog_manager.cc:5495] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84c51e48bc3c46268daa27d0cf9785d0 (127.11.207.130). New cstate: current_term: 1 leader_uuid: "84c51e48bc3c46268daa27d0cf9785d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c51e48bc3c46268daa27d0cf9785d0" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 45391 } health_report { overall_health: HEALTHY } } }
I20260812 21:12:38.160847 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.158166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.11.207.130:45391)
0812 21:12:38.158222 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.158224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.158233 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:12:38.158295 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.158296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.158318 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.158381 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:12:38.158524 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.159246 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.160087 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.160301 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.160383 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.160537 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:12:38.160647 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:12:38.160721 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.160745 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.160769 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:12:38.162669 15642 raft_consensus.cc:1261] T e4881e649fba4b529ce35d3e8c9ad361 P d8caef4dbb79421ea75beacff6c7e4f4 [term 1 FOLLOWER]: Refusing update from remote peer 84c51e48bc3c46268daa27d0cf9785d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:38.162998 16095 raft_consensus.cc:1261] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 1 FOLLOWER]: Refusing update from remote peer 84c51e48bc3c46268daa27d0cf9785d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:38.163043 16224 consensus_queue.cc:1035] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8caef4dbb79421ea75beacff6c7e4f4" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:38.163341 16224 consensus_queue.cc:1035] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd9e99e286cc49f9a9e5ccfab5d0820d" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 39353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:38.165499 16228 mvcc.cc:205] Tried to move back new op lower bound from 7317787271831683072 to 7317787271678046208. Current Snapshot: MvccSnapshot[applied={T|T < 7317787271831683072}]
I20260812 21:12:38.165835 16234 mvcc.cc:205] Tried to move back new op lower bound from 7317787271831683072 to 7317787271678046208. Current Snapshot: MvccSnapshot[applied={T|T < 7317787271831683072}]
I20260812 21:12:38.168538 15382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:38.168628 15382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:38.169359 15382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2159bb907c83463d92928e4fbe327be5: Sending DeleteTablet for 3 replicas of tablet e4881e649fba4b529ce35d3e8c9ad361
I20260812 21:12:38.169678 16060 tablet_service.cc:1514] Processing DeleteTablet for tablet e4881e649fba4b529ce35d3e8c9ad361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:38 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:55110
I20260812 21:12:38.169718 15600 tablet_service.cc:1514] Processing DeleteTablet for tablet e4881e649fba4b529ce35d3e8c9ad361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:38 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:58680
I20260812 21:12:38.170022 16246 tablet_replica.cc:330] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: stopping tablet replica
I20260812 21:12:38.170061 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet e4881e649fba4b529ce35d3e8c9ad361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:38 UTC) from {username='kudu', principal='kudu/127.11.207.190@KRBTEST.COM'} at 127.0.0.1:54340
I20260812 21:12:38.170162 16246 raft_consensus.cc:2229] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:38.170269 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15494
I20260812 21:12:38.170290 16246 raft_consensus.cc:2258] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:38.182566 16250 tablet_replica.cc:330] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: stopping tablet replica
I20260812 21:12:38.182741 16246 ts_tablet_manager.cc:1905] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:38.182901 16250 raft_consensus.cc:2229] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:12:38.183108 16250 raft_consensus.cc:2258] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:38.184610 16250 ts_tablet_manager.cc:1905] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:38.184913 16246 ts_tablet_manager.cc:1918] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:38.185020 16246 log.cc:1196] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/e4881e649fba4b529ce35d3e8c9ad361
I20260812 21:12:38.185384 16246 ts_tablet_manager.cc:1939] T e4881e649fba4b529ce35d3e8c9ad361 P dd9e99e286cc49f9a9e5ccfab5d0820d: Deleting consensus metadata
I20260812 21:12:38.185957 15355 catalog_manager.cc:4841] TS dd9e99e286cc49f9a9e5ccfab5d0820d (127.11.207.131:39353): tablet e4881e649fba4b529ce35d3e8c9ad361 (table test-table [id=fe493205464740acb6d88b608cfc7121]) successfully deleted
I20260812 21:12:38.186549 16250 ts_tablet_manager.cc:1918] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:38.186630 16250 log.cc:1196] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/e4881e649fba4b529ce35d3e8c9ad361
I20260812 21:12:38.186892 16250 ts_tablet_manager.cc:1939] T e4881e649fba4b529ce35d3e8c9ad361 P 84c51e48bc3c46268daa27d0cf9785d0: Deleting consensus metadata
I20260812 21:12:38.187341 15357 catalog_manager.cc:4841] TS 84c51e48bc3c46268daa27d0cf9785d0 (127.11.207.130:45391): tablet e4881e649fba4b529ce35d3e8c9ad361 (table test-table [id=fe493205464740acb6d88b608cfc7121]) successfully deleted
W20260812 21:12:38.190771 15361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:45715 (error 108)
I20260812 21:12:38.190991 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15737
I20260812 21:12:38.200726 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.200511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:45715 (local address 127.0.0.1:58694)
0812 21:12:38.200588 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.200678 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 21:12:38.200887 15361 catalog_manager.cc:4584] TS d8caef4dbb79421ea75beacff6c7e4f4 (127.11.207.129:45715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4881e649fba4b529ce35d3e8c9ad361: Network error: Client connection negotiation failed: client connection to 127.11.207.129:45715: connect: Connection refused (error 111)
I20260812 21:12:38.206897 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15967
I20260812 21:12:38.220181 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 15330
2026-08-12T21:12:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2350 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T21:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:38Z Disabled control of system clock
I20260812 21:12:38.258519 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38981
--webserver_interface=127.11.207.190
--webserver_port=0
--builtin_ntp_servers=127.11.207.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:38.340420 16259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:38.340655 16259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:38.340677 16259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:38.341849 16259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:38.341897 16259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:38.341908 16259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:38.341921 16259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:38.341933 16259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:38.343384 16259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:38981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:38.343637 16259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:38.343933 16259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:38.348905 16265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:38.348913 16259 server_base.cc:1034] running on GCE node
W20260812 21:12:38.348881 16268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:38.348881 16270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:38.349426 16259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:38.349699 16259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:38.350838 16259 hybrid_clock.cc:648] HybridClock initialized: now 1786569158350813 us; error 48 us; skew 500 ppm
I20260812 21:12:38.352044 16259 webserver.cc:460] Webserver started at http://127.11.207.190:43877/ using document root <none> and password file <none>
I20260812 21:12:38.352327 16259 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:38.352427 16259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:38.352566 16259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:38.353649 16259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "e692b05972124854a00847b30fe7e53d"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "05c8fb263ef63727f27356e52fbf7581"
server_key_iv: "490df2a9ec4f19eebf88c15f12f3a40f"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.353989 16259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "e692b05972124854a00847b30fe7e53d"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "05c8fb263ef63727f27356e52fbf7581"
server_key_iv: "490df2a9ec4f19eebf88c15f12f3a40f"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.355401 16259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:38.356165 16278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.356374 16259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:38.356470 16259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "e692b05972124854a00847b30fe7e53d"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "05c8fb263ef63727f27356e52fbf7581"
server_key_iv: "490df2a9ec4f19eebf88c15f12f3a40f"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.356539 16259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:38.388732 16259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:38.389087 16259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:38.389226 16259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:38.393163 16259 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:38981
I20260812 21:12:38.393208 16369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:38981 every 8 connection(s)
I20260812 21:12:38.393582 16259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:38.394596 16373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:38.397181 16373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d: Bootstrap starting.
I20260812 21:12:38.397794 16373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:38.398108 16373 log.cc:824] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:38.398845 16373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d: No bootstrap required, opened a new log
I20260812 21:12:38.400569 16373 raft_consensus.cc:348] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e692b05972124854a00847b30fe7e53d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38981 } }
I20260812 21:12:38.400727 16373 raft_consensus.cc:374] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:38.400753 16373 raft_consensus.cc:729] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e692b05972124854a00847b30fe7e53d, State: Initialized, Role: FOLLOWER
I20260812 21:12:38.400903 16373 consensus_queue.cc:260] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e692b05972124854a00847b30fe7e53d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38981 } }
I20260812 21:12:38.400972 16373 raft_consensus.cc:388] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:38.401007 16373 raft_consensus.cc:482] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:38.401062 16373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.401616 16373 raft_consensus.cc:504] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e692b05972124854a00847b30fe7e53d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38981 } }
I20260812 21:12:38.401722 16373 leader_election.cc:302] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e692b05972124854a00847b30fe7e53d; no voters:
I20260812 21:12:38.401891 16373 leader_election.cc:288] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:38.401943 16378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:38.402117 16378 raft_consensus.cc:686] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [term 1 LEADER]: Becoming Leader. State: Replica: e692b05972124854a00847b30fe7e53d, State: Running, Role: LEADER
I20260812 21:12:38.402158 16373 sys_catalog.cc:564] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:38.402226 16378 consensus_queue.cc:237] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e692b05972124854a00847b30fe7e53d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38981 } }
I20260812 21:12:38.402676 16382 sys_catalog.cc:455] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e692b05972124854a00847b30fe7e53d. Latest consensus state: current_term: 1 leader_uuid: "e692b05972124854a00847b30fe7e53d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e692b05972124854a00847b30fe7e53d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38981 } } }
I20260812 21:12:38.402752 16382 sys_catalog.cc:458] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:38.402860 16380 sys_catalog.cc:455] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e692b05972124854a00847b30fe7e53d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e692b05972124854a00847b30fe7e53d" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 38981 } } }
I20260812 21:12:38.402968 16389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:38.403004 16380 sys_catalog.cc:458] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:38.403491 16389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:38.403580 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16259
I20260812 21:12:38.403714 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:38.403911 12094 external_mini_cluster.cc:1428] Setting key 2fe2d10c14dc1d0dd8597ccf05955fab
I20260812 21:12:38.405143 16389 catalog_manager.cc:1347] Generated new cluster ID: 93f361a047984f69bf58e0de5f996b2d
I20260812 21:12:38.405202 16389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:38.410017 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.404590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.11.207.190:38981)
0812 21:12:38.404829 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.404837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.404871 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:12:38.404988 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.404995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.405063 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.405218 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:38.405441 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.406780 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.407818 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.408484 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.408610 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.409293 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:38.409297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:38.409297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:38.409514 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:38.409557 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:38.409572 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:38.409644 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.409715 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.409796 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260812 21:12:38.418123 16389 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:38.418623 16389 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:38.426783 16389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d: Generated new TSK 0
I20260812 21:12:38.426963 16389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:38.435416 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.129:0
--local_ip_for_outbound_sockets=127.11.207.129
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38981
--builtin_ntp_servers=127.11.207.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:12:38.521166 16412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:38.521376 16412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:38.521399 16412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:38.522543 16412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:38.522586 16412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:38.522611 16412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.129
I20260812 21:12:38.523967 16412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.207.129
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.129
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:38.524209 16412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:38.524554 16412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:38.530524 16423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:38.530768 16431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:38.530790 16412 server_base.cc:1034] running on GCE node
W20260812 21:12:38.530583 16429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:38.531021 16412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:38.531266 16412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:38.532433 16412 hybrid_clock.cc:648] HybridClock initialized: now 1786569158532414 us; error 31 us; skew 500 ppm
I20260812 21:12:38.533742 16412 webserver.cc:460] Webserver started at http://127.11.207.129:46611/ using document root <none> and password file <none>
I20260812 21:12:38.534013 16412 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:38.534068 16412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:38.534214 16412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:38.535338 16412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data/instance:
uuid: "e542e45adf2e4da8a9fb9e6cc3aca226"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "8da9104a531e2ae563ca089dfcc16ca3"
server_key_iv: "4e2c70b9499204263a7afeb4549318ab"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.535719 16412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance:
uuid: "e542e45adf2e4da8a9fb9e6cc3aca226"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "8da9104a531e2ae563ca089dfcc16ca3"
server_key_iv: "4e2c70b9499204263a7afeb4549318ab"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.537204 16412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:12:38.538067 16445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.538249 16412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:38.538335 16412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal
uuid: "e542e45adf2e4da8a9fb9e6cc3aca226"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "8da9104a531e2ae563ca089dfcc16ca3"
server_key_iv: "4e2c70b9499204263a7afeb4549318ab"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.538415 16412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:38.551059 16412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:38.551434 16412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:38.551585 16412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:38.551844 16412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:38.552196 16412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:38.552233 16412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.552289 16412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:38.552317 16412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.558784 16412 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.129:35987
I20260812 21:12:38.558853 16622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.129:35987 every 8 connection(s)
I20260812 21:12:38.559209 16412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/data/info.pb
I20260812 21:12:38.561612 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16412
I20260812 21:12:38.561738 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-0/wal/instance
I20260812 21:12:38.561980 12094 external_mini_cluster.cc:1428] Setting key a7833a60793400cf49e022b7d6eb4689
I20260812 21:12:38.562995 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.130:0
--local_ip_for_outbound_sockets=127.11.207.130
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38981
--builtin_ntp_servers=127.11.207.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:12:38.565243 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.560092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:35621 (local address 127.11.207.190:38981)
0812 21:12:38.560160 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.560162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.560475 (+ 313us) server_negotiation.cc:407] Connection header received
0812 21:12:38.561040 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.561041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.561055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.561111 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:38.561959 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.562527 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.563653 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.563908 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.563981 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.564992 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:38.564993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:38.564993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:38.564997 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:38.565014 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:38.565019 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:38.565062 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.565144 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.565183 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:12:38.565490 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.560081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38981 (local address 127.11.207.129:35621)
0812 21:12:38.560293 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.560384 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.560936 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.561178 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.561182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.561191 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:38.561857 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.561866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.562637 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.562638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.563563 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.563566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.564015 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.564016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.564149 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.564828 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:38.564829 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:38.564872 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:38.564904 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:38.565072 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:38.565077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:38.565081 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.565146 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.565259 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260812 21:12:38.566121 16623 heartbeater.cc:347] Connected to a master server at 127.11.207.190:38981
I20260812 21:12:38.566220 16623 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:38.566409 16623 heartbeater.cc:511] Master 127.11.207.190:38981 requested a full tablet report, sending...
I20260812 21:12:38.566949 16303 ts_manager.cc:194] Registered new tserver with Master: e542e45adf2e4da8a9fb9e6cc3aca226 (127.11.207.129:35987)
I20260812 21:12:38.567637 16303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.129:35621
W20260812 21:12:38.648084 16627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:38.648310 16627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:38.648334 16627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:38.649538 16627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:38.649587 16627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:38.649614 16627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.130
I20260812 21:12:38.650996 16627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.207.130
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.130
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:38.651235 16627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:38.651548 16627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:38.656651 16634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:38.656718 16637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:38.656663 16635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:38.656694 16627 server_base.cc:1034] running on GCE node
I20260812 21:12:38.657109 16627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:38.657351 16627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:38.658488 16627 hybrid_clock.cc:648] HybridClock initialized: now 1786569158658474 us; error 25 us; skew 500 ppm
I20260812 21:12:38.659693 16627 webserver.cc:460] Webserver started at http://127.11.207.130:44097/ using document root <none> and password file <none>
I20260812 21:12:38.659920 16627 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:38.659969 16627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:38.660099 16627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:38.661201 16627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data/instance:
uuid: "6e8c603b3b7340d983ecf3b371d631a3"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "8d8dfdc6a2748a2bc498c26756af7a92"
server_key_iv: "267d0b987480516edbd123a12e339143"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.661535 16627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance:
uuid: "6e8c603b3b7340d983ecf3b371d631a3"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "8d8dfdc6a2748a2bc498c26756af7a92"
server_key_iv: "267d0b987480516edbd123a12e339143"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.662753 16627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:12:38.663400 16648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.663568 16627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:38.663648 16627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal
uuid: "6e8c603b3b7340d983ecf3b371d631a3"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "8d8dfdc6a2748a2bc498c26756af7a92"
server_key_iv: "267d0b987480516edbd123a12e339143"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.663717 16627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:38.678898 16627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:38.679255 16627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:38.679406 16627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:38.679653 16627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:38.680001 16627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:38.680037 16627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.680066 16627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:38.680087 16627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.686084 16627 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.130:40567
I20260812 21:12:38.686127 16847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.130:40567 every 8 connection(s)
I20260812 21:12:38.686472 16627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/data/info.pb
I20260812 21:12:38.689435 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16627
I20260812 21:12:38.689546 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal/instance
I20260812 21:12:38.689782 12094 external_mini_cluster.cc:1428] Setting key a7a7d7ec885ea001eeb2e84d7c8550b8
I20260812 21:12:38.690747 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.207.131:0
--local_ip_for_outbound_sockets=127.11.207.131
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38981
--builtin_ntp_servers=127.11.207.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:12:38.692031 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.687208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.130:51833 (local address 127.11.207.190:38981)
0812 21:12:38.687252 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.687255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.687492 (+ 237us) server_negotiation.cc:407] Connection header received
0812 21:12:38.687978 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.687979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.687990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.688027 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:38.688816 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.689346 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.690633 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.690857 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.690921 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.691765 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:38.691766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:38.691766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:38.691771 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:38.691787 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:38.691791 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:38.691839 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.691915 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.691946 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:12:38.692318 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.687181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38981 (local address 127.11.207.130:51833)
0812 21:12:38.687375 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.687407 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.687883 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.688088 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.688091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.688099 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:38.688726 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.688733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.689434 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.689435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.690551 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.690554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.690985 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.690986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.691110 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.691621 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:38.691622 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:38.691653 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:38.691676 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:38.691844 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:38.691850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:38.691853 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.691913 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.692018 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260812 21:12:38.692763 16848 heartbeater.cc:347] Connected to a master server at 127.11.207.190:38981
I20260812 21:12:38.692857 16848 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:38.693032 16848 heartbeater.cc:511] Master 127.11.207.190:38981 requested a full tablet report, sending...
I20260812 21:12:38.693395 16303 ts_manager.cc:194] Registered new tserver with Master: 6e8c603b3b7340d983ecf3b371d631a3 (127.11.207.130:40567)
I20260812 21:12:38.693754 16303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.130:51833
W20260812 21:12:38.774427 16851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:38.774626 16851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:38.774646 16851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:38.775729 16851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:38.775761 16851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:38.775784 16851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.207.131
I20260812 21:12:38.777118 16851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.207.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.207.131
--webserver_port=0
--tserver_master_addrs=127.11.207.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.207.131
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:38.777312 16851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:38.777552 16851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:38.782519 16865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:38.782513 16861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:38.782513 16859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:38.782817 16851 server_base.cc:1034] running on GCE node
I20260812 21:12:38.783031 16851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:38.783283 16851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:38.784417 16851 hybrid_clock.cc:648] HybridClock initialized: now 1786569158784393 us; error 30 us; skew 500 ppm
I20260812 21:12:38.785777 16851 webserver.cc:460] Webserver started at http://127.11.207.131:37797/ using document root <none> and password file <none>
I20260812 21:12:38.786008 16851 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:38.786059 16851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:38.786191 16851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:38.787257 16851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data/instance:
uuid: "abb25a71a7094cd496aa10743ccbe025"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "ec09f6ff00056562aa5f478dedb84324"
server_key_iv: "96035fdb0c582ff803a4087dbe42a2ed"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.787591 16851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance:
uuid: "abb25a71a7094cd496aa10743ccbe025"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "ec09f6ff00056562aa5f478dedb84324"
server_key_iv: "96035fdb0c582ff803a4087dbe42a2ed"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.788940 16851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:38.789620 16877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.789795 16851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:38.789881 16851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal
uuid: "abb25a71a7094cd496aa10743ccbe025"
format_stamp: "Formatted at 2026-08-12 21:12:38 on dist-test-slave-lkx2"
server_key: "ec09f6ff00056562aa5f478dedb84324"
server_key_iv: "96035fdb0c582ff803a4087dbe42a2ed"
server_key_version: "encryptionkey@0"
I20260812 21:12:38.789945 16851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:38.800338 16851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:38.800661 16851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:38.800800 16851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:38.801015 16851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:12:38.801324 16851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:12:38.801358 16851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.801386 16851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:12:38.801409 16851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:38.807408 16851 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.131:36157
I20260812 21:12:38.807457 17068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.131:36157 every 8 connection(s)
I20260812 21:12:38.807814 16851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/data/info.pb
I20260812 21:12:38.813544 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.808717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.131:42419 (local address 127.11.207.190:38981)
0812 21:12:38.808786 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.808788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.809038 (+ 250us) server_negotiation.cc:407] Connection header received
0812 21:12:38.809585 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.809586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.809609 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.809662 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:38.810392 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.810935 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.811939 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.812145 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.812203 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.813191 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:38.813193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:38.813193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:38.813199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:38.813219 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:38.813223 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:38.813275 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.813378 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.813427 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:12:38.813730 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.808724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:38981 (local address 127.11.207.131:42419)
0812 21:12:38.808913 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.808949 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.809478 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.809735 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.809740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.809750 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:12:38.810283 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.810288 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.811029 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.811029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.811848 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.811851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.812216 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.812217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.812377 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.812998 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:12:38.812998 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:12:38.813031 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:12:38.813056 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:12:38.813284 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:12:38.813290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:12:38.813294 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.813373 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.813494 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260812 21:12:38.814240 17069 heartbeater.cc:347] Connected to a master server at 127.11.207.190:38981
I20260812 21:12:38.814339 17069 heartbeater.cc:464] Registering TS with master...
I20260812 21:12:38.814522 17069 heartbeater.cc:511] Master 127.11.207.190:38981 requested a full tablet report, sending...
I20260812 21:12:38.814888 16303 ts_manager.cc:194] Registered new tserver with Master: abb25a71a7094cd496aa10743ccbe025 (127.11.207.131:36157)
I20260812 21:12:38.815238 16303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.207.131:42419
I20260812 21:12:38.817214 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 16851
I20260812 21:12:38.817314 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal/instance
I20260812 21:12:38.817499 12094 external_mini_cluster.cc:1428] Setting key c623dcd52a2f4f4880756da7c792690e
I20260812 21:12:38.818563 12094 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:12:38.822754 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.819957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.11.207.190:38981)
0812 21:12:38.820026 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.820028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.820135 (+ 107us) server_negotiation.cc:407] Connection header received
0812 21:12:38.820183 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.820184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.820195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.820242 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:38.820427 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.820937 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.821699 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.821886 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.821939 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.822506 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:38.822507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:38.822507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:38.822512 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:38.822527 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:38.822531 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:38.822568 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.822606 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.822631 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 21:12:38.824208 16303 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:12:38.825899 16303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:12:38.832237 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.828075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:36157 (local address 127.0.0.1:55302)
0812 21:12:38.828627 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.828636 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.828676 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.829020 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.829021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.829023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.829070 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.829071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.829981 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.829982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.831142 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.831145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.831939 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.831940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.832018 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.832033 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.832097 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.832150 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":52,"threads_started":1}
I20260812 21:12:38.832367 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.828434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.11.207.129:35987)
0812 21:12:38.828633 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.828638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.828651 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:12:38.828717 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.828719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.828735 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.828816 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.829199 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.830353 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.831345 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.832022 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.832100 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.832187 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.832214 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.832254 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":95,"threads_started":1}
I20260812 21:12:38.832526 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.828083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40567 (local address 127.0.0.1:59266)
0812 21:12:38.828495 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.828530 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.828738 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.829063 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.829064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.829065 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.829101 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.829102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.830397 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.830398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.831467 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.831469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.832303 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.832304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.832378 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.832382 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.832427 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.832465 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":29,"thread_start_us":105,"threads_started":1}
I20260812 21:12:38.832546 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.828284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.11.207.130:40567)
0812 21:12:38.828502 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.828508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.828603 (+ 95us) server_negotiation.cc:407] Connection header received
0812 21:12:38.828941 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.828944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.828961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.829065 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.829152 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.830302 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.831547 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.832232 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.832307 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.832409 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.832437 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.832476 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
I20260812 21:12:38.832659 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.828303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:35987 (local address 127.0.0.1:52678)
0812 21:12:38.828559 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.828569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.828633 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.828859 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.828862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.828865 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.828942 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.828944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.830436 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.830437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.831272 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.831275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.832095 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.832096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.832147 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.832150 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.832185 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.832225 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
I20260812 21:12:38.833014 17086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.828191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.11.207.131:36157)
0812 21:12:38.828863 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.828867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.828876 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:12:38.828915 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.828916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.828930 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.829140 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.829205 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.829905 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.831250 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.831857 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.831921 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.832014 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.832889 (+ 875us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.832935 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":116,"threads_started":1}
I20260812 21:12:38.833364 16735 tablet_service.cc:1467] Processing CreateTablet for tablet 705e37533b8f42f9a6802d38aa3bd824 (DEFAULT_TABLE table=test-table [id=edd39c65d128457b870e72e14a9d9faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:38.833503 16517 tablet_service.cc:1467] Processing CreateTablet for tablet 705e37533b8f42f9a6802d38aa3bd824 (DEFAULT_TABLE table=test-table [id=edd39c65d128457b870e72e14a9d9faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:38.833694 16735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705e37533b8f42f9a6802d38aa3bd824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:38.833783 16517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705e37533b8f42f9a6802d38aa3bd824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:38.836454 17096 tablet_bootstrap.cc:492] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Bootstrap starting.
I20260812 21:12:38.836591 16967 tablet_service.cc:1467] Processing CreateTablet for tablet 705e37533b8f42f9a6802d38aa3bd824 (DEFAULT_TABLE table=test-table [id=edd39c65d128457b870e72e14a9d9faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:12:38.836808 17100 tablet_bootstrap.cc:492] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: Bootstrap starting.
I20260812 21:12:38.836957 16967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705e37533b8f42f9a6802d38aa3bd824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:38.837188 17096 tablet_bootstrap.cc:654] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:38.837474 17100 tablet_bootstrap.cc:654] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:38.837522 17096 log.cc:824] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:38.837805 17100 log.cc:824] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:38.838447 17096 tablet_bootstrap.cc:492] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: No bootstrap required, opened a new log
I20260812 21:12:38.838507 17096 ts_tablet_manager.cc:1397] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:12:38.839790 17106 tablet_bootstrap.cc:492] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Bootstrap starting.
I20260812 21:12:38.840176 17100 tablet_bootstrap.cc:492] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: No bootstrap required, opened a new log
I20260812 21:12:38.840242 17100 ts_tablet_manager.cc:1397] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 21:12:38.840461 17106 tablet_bootstrap.cc:654] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:38.840673 17096 raft_consensus.cc:348] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.840796 17106 log.cc:824] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:38.840857 17096 raft_consensus.cc:374] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:38.840899 17096 raft_consensus.cc:729] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8c603b3b7340d983ecf3b371d631a3, State: Initialized, Role: FOLLOWER
I20260812 21:12:38.841012 17096 consensus_queue.cc:260] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.841284 17096 ts_tablet_manager.cc:1428] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 21:12:38.841305 16848 heartbeater.cc:503] Master 127.11.207.190:38981 was elected leader, sending a full tablet report...
I20260812 21:12:38.841804 17106 tablet_bootstrap.cc:492] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: No bootstrap required, opened a new log
I20260812 21:12:38.841866 17106 ts_tablet_manager.cc:1397] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:12:38.842639 17100 raft_consensus.cc:348] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.842794 17100 raft_consensus.cc:374] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:38.842818 17100 raft_consensus.cc:729] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e542e45adf2e4da8a9fb9e6cc3aca226, State: Initialized, Role: FOLLOWER
I20260812 21:12:38.842912 17100 consensus_queue.cc:260] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.843140 17100 ts_tablet_manager.cc:1428] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 21:12:38.843214 16623 heartbeater.cc:503] Master 127.11.207.190:38981 was elected leader, sending a full tablet report...
I20260812 21:12:38.844173 17106 raft_consensus.cc:348] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.844292 17106 raft_consensus.cc:374] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:38.844312 17106 raft_consensus.cc:729] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb25a71a7094cd496aa10743ccbe025, State: Initialized, Role: FOLLOWER
I20260812 21:12:38.844435 17106 consensus_queue.cc:260] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.844643 17106 ts_tablet_manager.cc:1428] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 21:12:38.844709 17069 heartbeater.cc:503] Master 127.11.207.190:38981 was elected leader, sending a full tablet report...
I20260812 21:12:38.854118 17110 raft_consensus.cc:482] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:12:38.854228 17110 raft_consensus.cc:504] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.854544 17110 leader_election.cc:288] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e8c603b3b7340d983ecf3b371d631a3 (127.11.207.130:40567), abb25a71a7094cd496aa10743ccbe025 (127.11.207.131:36157)
I20260812 21:12:38.857630 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.854710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.131:36157 (local address 127.11.207.129:54167)
0812 21:12:38.854765 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.854778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.854835 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.854979 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.854980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.854982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.855064 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.855066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.855925 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.855927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.856793 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.856796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.857386 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.857387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.857458 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.857463 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.857523 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.857566 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":22}
I20260812 21:12:38.857646 17086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.854801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:54167 (local address 127.11.207.131:36157)
0812 21:12:38.854847 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.854849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.854859 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:12:38.854892 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.854893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.854902 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.854967 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.855181 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.855837 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.856877 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.857315 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.857370 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.857377 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.857536 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.857577 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:12:38.857805 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.854710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.130:40567 (local address 127.11.207.129:37405)
0812 21:12:38.854905 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.854914 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:12:38.854969 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:12:38.855158 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:12:38.855160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:12:38.855162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:12:38.855271 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.855273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.856034 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.856035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.856993 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:12:38.856996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.857543 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.857544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:12:38.857620 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.857624 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:12:38.857690 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:12:38.857727 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":16,"thread_start_us":115,"threads_started":1}
I20260812 21:12:38.857807 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.854838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.207.129:37405 (local address 127.11.207.130:40567)
0812 21:12:38.854878 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.854880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.854971 (+ 91us) server_negotiation.cc:407] Connection header received
0812 21:12:38.855020 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.855021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.855031 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.855077 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:12:38.855358 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.855941 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.857074 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.857469 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.857533 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.857540 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.857704 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.857743 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:12:38.858142 16772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705e37533b8f42f9a6802d38aa3bd824" candidate_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8c603b3b7340d983ecf3b371d631a3" is_pre_election: true
I20260812 21:12:38.858174 16995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705e37533b8f42f9a6802d38aa3bd824" candidate_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb25a71a7094cd496aa10743ccbe025" is_pre_election: true
I20260812 21:12:38.858282 16772 raft_consensus.cc:2454] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e542e45adf2e4da8a9fb9e6cc3aca226 in term 0.
I20260812 21:12:38.858309 16995 raft_consensus.cc:2454] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e542e45adf2e4da8a9fb9e6cc3aca226 in term 0.
I20260812 21:12:38.858511 16449 leader_election.cc:302] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8c603b3b7340d983ecf3b371d631a3, e542e45adf2e4da8a9fb9e6cc3aca226; no voters:
I20260812 21:12:38.858637 17110 raft_consensus.cc:2781] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:12:38.858703 17110 raft_consensus.cc:482] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:12:38.858750 17110 raft_consensus.cc:3037] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.859455 17110 raft_consensus.cc:504] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.859570 17110 leader_election.cc:288] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [CANDIDATE]: Term 1 election: Requested vote from peers 6e8c603b3b7340d983ecf3b371d631a3 (127.11.207.130:40567), abb25a71a7094cd496aa10743ccbe025 (127.11.207.131:36157)
I20260812 21:12:38.859732 16772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705e37533b8f42f9a6802d38aa3bd824" candidate_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8c603b3b7340d983ecf3b371d631a3"
I20260812 21:12:38.859781 16995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705e37533b8f42f9a6802d38aa3bd824" candidate_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb25a71a7094cd496aa10743ccbe025"
I20260812 21:12:38.859850 16995 raft_consensus.cc:3037] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.859839 16772 raft_consensus.cc:3037] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:38.860647 16772 raft_consensus.cc:2454] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e542e45adf2e4da8a9fb9e6cc3aca226 in term 1.
I20260812 21:12:38.860661 16995 raft_consensus.cc:2454] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e542e45adf2e4da8a9fb9e6cc3aca226 in term 1.
I20260812 21:12:38.860803 16449 leader_election.cc:302] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8c603b3b7340d983ecf3b371d631a3, e542e45adf2e4da8a9fb9e6cc3aca226; no voters:
I20260812 21:12:38.860890 17110 raft_consensus.cc:2781] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:38.861029 17110 raft_consensus.cc:686] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [term 1 LEADER]: Becoming Leader. State: Replica: e542e45adf2e4da8a9fb9e6cc3aca226, State: Running, Role: LEADER
I20260812 21:12:38.861111 17110 consensus_queue.cc:237] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } }
I20260812 21:12:38.861943 16303 catalog_manager.cc:5495] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 reported cstate change: term changed from 0 to 1, leader changed from <none> to e542e45adf2e4da8a9fb9e6cc3aca226 (127.11.207.129). New cstate: current_term: 1 leader_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e542e45adf2e4da8a9fb9e6cc3aca226" member_type: VOTER last_known_addr { host: "127.11.207.129" port: 35987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 } health_report { overall_health: UNKNOWN } } }
I20260812 21:12:38.867229 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.864946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.11.207.129:35987)
0812 21:12:38.864995 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:38.864997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:38.865004 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:12:38.865064 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:38.865064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:38.865081 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:38.865137 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:12:38.865243 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.865686 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.866418 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:38.866619 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:38.866710 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:38.866880 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:12:38.866994 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:12:38.867067 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:38.867128 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:12:38.867148 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:12:38.869128 16772 raft_consensus.cc:1261] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 1 FOLLOWER]: Refusing update from remote peer e542e45adf2e4da8a9fb9e6cc3aca226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:38.869128 16995 raft_consensus.cc:1261] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 1 FOLLOWER]: Refusing update from remote peer e542e45adf2e4da8a9fb9e6cc3aca226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:12:38.869483 17114 consensus_queue.cc:1035] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e8c603b3b7340d983ecf3b371d631a3" member_type: VOTER last_known_addr { host: "127.11.207.130" port: 40567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:38.869715 17110 consensus_queue.cc:1035] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb25a71a7094cd496aa10743ccbe025" member_type: VOTER last_known_addr { host: "127.11.207.131" port: 36157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:12:38.872061 17120 mvcc.cc:205] Tried to move back new op lower bound from 7317787274725203968 to 7317787274695770112. Current Snapshot: MvccSnapshot[applied={T|T < 7317787274725203968}]
I20260812 21:12:38.874817 16303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:38.874917 16303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:12:38.875718 16303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e692b05972124854a00847b30fe7e53d: Sending DeleteTablet for 3 replicas of tablet 705e37533b8f42f9a6802d38aa3bd824
I20260812 21:12:38.876103 16517 tablet_service.cc:1514] Processing DeleteTablet for tablet 705e37533b8f42f9a6802d38aa3bd824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:38 UTC) from {username='slave'} at 127.0.0.1:52678
I20260812 21:12:38.876109 16967 tablet_service.cc:1514] Processing DeleteTablet for tablet 705e37533b8f42f9a6802d38aa3bd824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:38 UTC) from {username='slave'} at 127.0.0.1:55302
I20260812 21:12:38.876462 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16412
I20260812 21:12:38.876482 16735 tablet_service.cc:1514] Processing DeleteTablet for tablet 705e37533b8f42f9a6802d38aa3bd824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:12:38 UTC) from {username='slave'} at 127.0.0.1:59266
I20260812 21:12:38.876683 17140 tablet_replica.cc:330] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: stopping tablet replica
I20260812 21:12:38.876731 17141 tablet_replica.cc:330] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: stopping tablet replica
I20260812 21:12:38.876835 17138 tablet_replica.cc:330] T 705e37533b8f42f9a6802d38aa3bd824 P e542e45adf2e4da8a9fb9e6cc3aca226: stopping tablet replica
I20260812 21:12:38.876857 17141 raft_consensus.cc:2229] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:38.876976 17141 raft_consensus.cc:2258] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:38.877197 17140 raft_consensus.cc:2229] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:12:38.879868 17140 raft_consensus.cc:2258] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:12:38.879984 17141 ts_tablet_manager.cc:1905] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:38.880450 17140 ts_tablet_manager.cc:1905] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:12:38.881735 17141 ts_tablet_manager.cc:1918] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:38.881815 17141 log.cc:1196] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-1/wal/wals/705e37533b8f42f9a6802d38aa3bd824
I20260812 21:12:38.882045 17141 ts_tablet_manager.cc:1939] T 705e37533b8f42f9a6802d38aa3bd824 P 6e8c603b3b7340d983ecf3b371d631a3: Deleting consensus metadata
I20260812 21:12:38.882175 17140 ts_tablet_manager.cc:1918] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:12:38.882262 17140 log.cc:1196] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Deleting WAL directory at /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568904794702-12094-0/minicluster-data/ts-2/wal/wals/705e37533b8f42f9a6802d38aa3bd824
I20260812 21:12:38.882567 17140 ts_tablet_manager.cc:1939] T 705e37533b8f42f9a6802d38aa3bd824 P abb25a71a7094cd496aa10743ccbe025: Deleting consensus metadata
I20260812 21:12:38.884943 16282 catalog_manager.cc:4841] TS 6e8c603b3b7340d983ecf3b371d631a3 (127.11.207.130:40567): tablet 705e37533b8f42f9a6802d38aa3bd824 (table test-table [id=edd39c65d128457b870e72e14a9d9faa]) successfully deleted
I20260812 21:12:38.885280 16286 catalog_manager.cc:4841] TS abb25a71a7094cd496aa10743ccbe025 (127.11.207.131:36157): tablet 705e37533b8f42f9a6802d38aa3bd824 (table test-table [id=edd39c65d128457b870e72e14a9d9faa]) successfully deleted
W20260812 21:12:38.889392 16282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.207.129:35987 (error 108)
I20260812 21:12:38.889559 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16627
I20260812 21:12:38.892952 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:38.892809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.129:35987 (local address 127.0.0.1:52700)
0812 21:12:38.892853 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:12:38.892902 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.129:35987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 21:12:38.893183 16282 catalog_manager.cc:4584] TS e542e45adf2e4da8a9fb9e6cc3aca226 (127.11.207.129:35987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 705e37533b8f42f9a6802d38aa3bd824: Network error: Client connection negotiation failed: client connection to 127.11.207.129:35987: connect: Connection refused (error 111)
I20260812 21:12:38.910261 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16851
I20260812 21:12:38.929149 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 16259
2026-08-12T21:12:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (707 ms)
[----------] 6 tests from EncryptionPolicyTest (9203 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:38 dist-test-slave-lkx2 krb5kdc[17153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:38 dist-test-slave-lkx2 krb5kdc[17153](info): set up 2 sockets
Aug 12 21:12:38 dist-test-slave-lkx2 krb5kdc[17153](info): commencing operation
krb5kdc: starting...
W20260812 21:12:40.133723 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:40 dist-test-slave-lkx2 krb5kdc[17153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:40.253072 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:40.335165 17170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:40.335397 17170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:40.335430 17170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:40.336745 17170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:40.336799 17170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:40.336818 17170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:40.336834 17170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:40.336849 17170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:40.338382 17170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:40.338629 17170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:40.338847 17170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:40.343855 17183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:40.343943 17180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:40.344020 17170 server_base.cc:1034] running on GCE node
W20260812 21:12:40.343866 17178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:40.344362 17170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:40.344731 17170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:40.345888 17170 hybrid_clock.cc:648] HybridClock initialized: now 1786569160345874 us; error 26 us; skew 500 ppm
Aug 12 21:12:40 dist-test-slave-lkx2 krb5kdc[17153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569160, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:40.351800 17170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:40.352267 17170 webserver.cc:460] Webserver started at http://127.0.0.1:45415/ using document root <none> and password file <none>
I20260812 21:12:40.352492 17170 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:40.352540 17170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:40.352659 17170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:40.353753 17170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "212418cac6884492ba26a45075e20b52"
format_stamp: "Formatted at 2026-08-12 21:12:40 on dist-test-slave-lkx2"
server_key: "cc45b8bb6c71e105c4411bdd76d1b942"
server_key_iv: "3a567303865f90eb16ae7b51bdda2dde"
server_key_version: "encryptionkey@0"
I20260812 21:12:40.354084 17170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "212418cac6884492ba26a45075e20b52"
format_stamp: "Formatted at 2026-08-12 21:12:40 on dist-test-slave-lkx2"
server_key: "cc45b8bb6c71e105c4411bdd76d1b942"
server_key_iv: "3a567303865f90eb16ae7b51bdda2dde"
server_key_version: "encryptionkey@0"
I20260812 21:12:40.355906 17170 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 21:12:40.356756 17192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:40.356976 17170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:12:40.357074 17170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "212418cac6884492ba26a45075e20b52"
format_stamp: "Formatted at 2026-08-12 21:12:40 on dist-test-slave-lkx2"
server_key: "cc45b8bb6c71e105c4411bdd76d1b942"
server_key_iv: "3a567303865f90eb16ae7b51bdda2dde"
server_key_version: "encryptionkey@0"
I20260812 21:12:40.357141 17170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:40.370517 17170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:40.372696 17170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:40.372851 17170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:40.377138 17170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35171
I20260812 21:12:40.377166 17286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35171 every 8 connection(s)
I20260812 21:12:40.377537 17170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:40.378337 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17170
I20260812 21:12:40.378456 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:40.378444 17287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:40.378691 12094 external_mini_cluster.cc:1428] Setting key e66f9291465bcb2fee6b31f75cfb9368
I20260812 21:12:40.381675 17287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52: Bootstrap starting.
I20260812 21:12:40.382404 17287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:40.382731 17287 log.cc:824] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:40.383441 17287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52: No bootstrap required, opened a new log
I20260812 21:12:40.385538 17287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212418cac6884492ba26a45075e20b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35171 } }
I20260812 21:12:40.385722 17287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 12 21:12:40 dist-test-slave-lkx2 krb5kdc[17153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:12:40.385759 17287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 212418cac6884492ba26a45075e20b52, State: Initialized, Role: FOLLOWER
I20260812 21:12:40.385895 17287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212418cac6884492ba26a45075e20b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35171 } }
I20260812 21:12:40.385983 17287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:40.386034 17287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:40.386085 17287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:40.386682 17287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212418cac6884492ba26a45075e20b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35171 } }
I20260812 21:12:40.386806 17287 leader_election.cc:302] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 212418cac6884492ba26a45075e20b52; no voters:
I20260812 21:12:40.386962 17287 leader_election.cc:288] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:40.387003 17299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:40.387111 17299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [term 1 LEADER]: Becoming Leader. State: Replica: 212418cac6884492ba26a45075e20b52, State: Running, Role: LEADER
I20260812 21:12:40.387217 17299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212418cac6884492ba26a45075e20b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35171 } }
I20260812 21:12:40.387248 17287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:40.387540 17299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 212418cac6884492ba26a45075e20b52. Latest consensus state: current_term: 1 leader_uuid: "212418cac6884492ba26a45075e20b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212418cac6884492ba26a45075e20b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35171 } } }
I20260812 21:12:40.387626 17299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:40.387552 17301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "212418cac6884492ba26a45075e20b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212418cac6884492ba26a45075e20b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35171 } } }
I20260812 21:12:40.387696 17301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52 [sys.catalog]: This master's current role is: LEADER
W20260812 21:12:40.388948 17328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:12:40.389171 17328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:12:40.389245 17311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:40.389394 17311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:40.390420 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:40.379269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45895 (local address 127.0.0.1:35171)
0812 21:12:40.379446 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:40.379456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:40.379484 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:12:40.379955 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:40.379962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:40.380010 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:40.380147 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:40.380973 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:40.381936 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:40.382979 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:40.383487 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:40.383583 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:40.386314 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:40.386319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:40.386319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:40.386330 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:40.388565 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:40.389102 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:40.389105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:40.389105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:40.389220 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:40.389484 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:40.389485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:40.389485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:40.389792 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:40.389951 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:40.390126 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 21:12:40.390231 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260812 21:12:40.390895 17311 catalog_manager.cc:1347] Generated new cluster ID: 231a207aeaeb473984e0a2979a18dc31
I20260812 21:12:40.390946 17311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:40.391607 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:40.406289 17311 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:40.406968 17311 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:40.407846 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:40.396699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55705 (local address 127.0.0.1:35171)
0812 21:12:40.396733 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:40.396735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:40.400530 (+ 3795us) server_negotiation.cc:407] Connection header received
0812 21:12:40.400589 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:40.400590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:40.400602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:40.400654 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:40.401263 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:40.402029 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:40.403001 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:40.403350 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:40.403412 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:40.404828 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:40.404830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:40.404830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:40.404838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:40.406458 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:40.406934 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:40.406935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:40.406936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:40.407006 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:40.407303 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:40.407304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:40.407304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:40.407433 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:40.407552 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:40.407697 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 21:12:40.407761 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:12:40.411644 17311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 212418cac6884492ba26a45075e20b52: Generated new TSK 0
I20260812 21:12:40.411849 17311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:40.414184 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17170
W20260812 21:12:40.439163 17149 connection.cc:535] client connection to 127.0.0.1:35171 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35171): Connection reset by peer (error 104)
2026-08-12T21:12:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:40 dist-test-slave-lkx2 krb5kdc[17349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:40 dist-test-slave-lkx2 krb5kdc[17349](info): set up 2 sockets
Aug 12 21:12:40 dist-test-slave-lkx2 krb5kdc[17349](info): commencing operation
krb5kdc: starting...
W20260812 21:12:41.641054 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:41 dist-test-slave-lkx2 krb5kdc[17349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:41.761001 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:41.845971 17366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:41.846184 17366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:41.846208 17366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:41.847402 17366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:41.847452 17366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:41.847465 17366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:41.847476 17366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:41.847486 17366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:41.849076 17366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:41.849321 17366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:41.849543 17366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:41.854583 17379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:41.854590 17383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:41.854660 17366 server_base.cc:1034] running on GCE node
W20260812 21:12:41.854720 17376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:41.855178 17366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:41.855464 17366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:41.856607 17366 hybrid_clock.cc:648] HybridClock initialized: now 1786569161856589 us; error 35 us; skew 500 ppm
Aug 12 21:12:41 dist-test-slave-lkx2 krb5kdc[17349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:41.859053 17366 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:41.859490 17366 webserver.cc:460] Webserver started at http://127.0.0.1:40447/ using document root <none> and password file <none>
I20260812 21:12:41.859714 17366 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:41.859758 17366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:41.859890 17366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:41.861037 17366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "050197e12cbf413f9f5b8c1652809f0a"
format_stamp: "Formatted at 2026-08-12 21:12:41 on dist-test-slave-lkx2"
server_key: "d78d537acb68b14aa38e6563ba52cf01"
server_key_iv: "8162cfb1ace50c44d490a53171ed53b9"
server_key_version: "encryptionkey@0"
I20260812 21:12:41.861366 17366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "050197e12cbf413f9f5b8c1652809f0a"
format_stamp: "Formatted at 2026-08-12 21:12:41 on dist-test-slave-lkx2"
server_key: "d78d537acb68b14aa38e6563ba52cf01"
server_key_iv: "8162cfb1ace50c44d490a53171ed53b9"
server_key_version: "encryptionkey@0"
I20260812 21:12:41.862643 17366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:12:41.863346 17398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:41.863478 17366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:41.863561 17366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "050197e12cbf413f9f5b8c1652809f0a"
format_stamp: "Formatted at 2026-08-12 21:12:41 on dist-test-slave-lkx2"
server_key: "d78d537acb68b14aa38e6563ba52cf01"
server_key_iv: "8162cfb1ace50c44d490a53171ed53b9"
server_key_version: "encryptionkey@0"
I20260812 21:12:41.863631 17366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:41.874825 17366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:41.876744 17366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:41.876880 17366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:41.880829 17366 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39031
I20260812 21:12:41.880860 17477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39031 every 8 connection(s)
I20260812 21:12:41.881199 17366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:41.882054 17478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:41.884330 17478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a: Bootstrap starting.
I20260812 21:12:41.884943 17478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:41.885178 17478 log.cc:824] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:41.885816 17478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a: No bootstrap required, opened a new log
I20260812 21:12:41.886215 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17366
I20260812 21:12:41.886322 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:41.886505 12094 external_mini_cluster.cc:1428] Setting key fda77950e1429b6089a44f499078e52b
I20260812 21:12:41.887758 17478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050197e12cbf413f9f5b8c1652809f0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39031 } }
I20260812 21:12:41.887913 17478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:41.887952 17478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 050197e12cbf413f9f5b8c1652809f0a, State: Initialized, Role: FOLLOWER
I20260812 21:12:41.888082 17478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050197e12cbf413f9f5b8c1652809f0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39031 } }
I20260812 21:12:41.888171 17478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:41.888218 17478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:41.888266 17478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:41.888845 17478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050197e12cbf413f9f5b8c1652809f0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39031 } }
I20260812 21:12:41.888935 17478 leader_election.cc:302] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 050197e12cbf413f9f5b8c1652809f0a; no voters:
I20260812 21:12:41.889088 17478 leader_election.cc:288] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:41.889128 17485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:41.889290 17485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [term 1 LEADER]: Becoming Leader. State: Replica: 050197e12cbf413f9f5b8c1652809f0a, State: Running, Role: LEADER
I20260812 21:12:41.889370 17478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:41.889432 17485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050197e12cbf413f9f5b8c1652809f0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39031 } }
I20260812 21:12:41.889823 17489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "050197e12cbf413f9f5b8c1652809f0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050197e12cbf413f9f5b8c1652809f0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39031 } } }
I20260812 21:12:41.889919 17489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:41.889811 17487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 050197e12cbf413f9f5b8c1652809f0a. Latest consensus state: current_term: 1 leader_uuid: "050197e12cbf413f9f5b8c1652809f0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050197e12cbf413f9f5b8c1652809f0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39031 } } }
I20260812 21:12:41.890196 17487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:41.890627 17498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:41.891175 17498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:41.892761 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:41.887012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54829 (local address 127.0.0.1:39031)
0812 21:12:41.887185 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:41.887193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:41.887219 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:12:41.887717 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:41.887723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:41.887768 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:41.887886 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:41.888152 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:41.889663 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:41.890732 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:41.891218 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:41.891325 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:41.892049 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:41.892053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:41.892053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:41.892241 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:41.892282 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:41.892298 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:41.892422 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:41.892514 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 21:12:41.892545 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260812 21:12:41.892765 17498 catalog_manager.cc:1347] Generated new cluster ID: fe22162e8a75495eb4f09f1d7d2cfbe4
I20260812 21:12:41.892841 17498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:41.893668 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:41.897737 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:41.895076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46277 (local address 127.0.0.1:39031)
0812 21:12:41.895125 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:41.895127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:41.895205 (+ 78us) server_negotiation.cc:407] Connection header received
0812 21:12:41.895263 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:41.895264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:41.895273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:41.895314 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:41.895426 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:41.895880 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:41.896632 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:41.896860 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:41.896915 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:41.897516 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:41.897517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:41.897517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:41.897521 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:41.897535 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:41.897539 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:41.897589 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:41.897654 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 21:12:41.897665 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:12:41.901664 17498 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:41.902143 17498 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:41.904978 17498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 050197e12cbf413f9f5b8c1652809f0a: Generated new TSK 0
I20260812 21:12:41.905145 17498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:41.908876 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17366
W20260812 21:12:41.927673 17342 connection.cc:535] client connection to 127.0.0.1:39031 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39031): Connection reset by peer (error 104)
2026-08-12T21:12:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:41 dist-test-slave-lkx2 krb5kdc[17530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:41 dist-test-slave-lkx2 krb5kdc[17530](info): set up 2 sockets
Aug 12 21:12:41 dist-test-slave-lkx2 krb5kdc[17530](info): commencing operation
krb5kdc: starting...
W20260812 21:12:43.113610 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:43 dist-test-slave-lkx2 krb5kdc[17530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:43.236016 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:43.319530 17547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:43.319756 17547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:43.319789 17547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:43.320969 17547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:43.321013 17547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:43.321031 17547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:43.321048 17547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:43.321063 17547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:43.322542 17547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:43.322796 17547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:43.323002 17547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:12:43.327907 17547 server_base.cc:1034] running on GCE node
W20260812 21:12:43.327891 17559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:43.327903 17560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:43.327893 17562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:43.328284 17547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:43.328617 17547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:43.329761 17547 hybrid_clock.cc:648] HybridClock initialized: now 1786569163329745 us; error 30 us; skew 500 ppm
Aug 12 21:12:43 dist-test-slave-lkx2 krb5kdc[17530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569163, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:43.332237 17547 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:43.332676 17547 webserver.cc:460] Webserver started at http://127.0.0.1:43363/ using document root <none> and password file <none>
I20260812 21:12:43.332870 17547 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:43.332916 17547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:43.333019 17547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:43.334091 17547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "c3b5223b19814cf2bcd4fffeeba2f13c"
format_stamp: "Formatted at 2026-08-12 21:12:43 on dist-test-slave-lkx2"
server_key: "f147cb4f35ddf81297eb3c9a929b626f"
server_key_iv: "78be0d493854b091a4e0893c94d10a84"
server_key_version: "encryptionkey@0"
I20260812 21:12:43.334414 17547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "c3b5223b19814cf2bcd4fffeeba2f13c"
format_stamp: "Formatted at 2026-08-12 21:12:43 on dist-test-slave-lkx2"
server_key: "f147cb4f35ddf81297eb3c9a929b626f"
server_key_iv: "78be0d493854b091a4e0893c94d10a84"
server_key_version: "encryptionkey@0"
I20260812 21:12:43.335705 17547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:12:43.336424 17571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:43.336571 17547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:43.336687 17547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "c3b5223b19814cf2bcd4fffeeba2f13c"
format_stamp: "Formatted at 2026-08-12 21:12:43 on dist-test-slave-lkx2"
server_key: "f147cb4f35ddf81297eb3c9a929b626f"
server_key_iv: "78be0d493854b091a4e0893c94d10a84"
server_key_version: "encryptionkey@0"
I20260812 21:12:43.336751 17547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:43.339933 17547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:43.342010 17547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:43.342146 17547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:43.346078 17547 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45869
I20260812 21:12:43.346119 17665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45869 every 8 connection(s)
I20260812 21:12:43.346429 17547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:43.347396 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:43.349776 17666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c: Bootstrap starting.
I20260812 21:12:43.350320 17666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:43.350551 17666 log.cc:824] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:43.351181 17666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c: No bootstrap required, opened a new log
I20260812 21:12:43.351593 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17547
I20260812 21:12:43.351722 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:43.351934 12094 external_mini_cluster.cc:1428] Setting key db6de1651ff7d238bdc116b0b8b14845
I20260812 21:12:43.353237 17666 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260812 21:12:43.353415 17666 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:43.353497 17666 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b5223b19814cf2bcd4fffeeba2f13c, State: Initialized, Role: FOLLOWER
I20260812 21:12:43.353637 17666 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260812 21:12:43.353724 17666 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:43.353749 17666 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:43.353832 17666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:43.354409 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:43.352456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55283 (local address 127.0.0.1:45869)
0812 21:12:43.352633 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:43.352641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:43.352694 (+ 53us) server_negotiation.cc:407] Connection header received
0812 21:12:43.353191 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:43.353196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:43.353220 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:43.353331 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:43.354000 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:43.354004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:43.354004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:43.354007 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:43.354047 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:43.354058 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:43.354111 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:43.354185 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 21:12:43.354217 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260812 21:12:43.354993 17666 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260812 21:12:43.355140 17666 leader_election.cc:302] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b5223b19814cf2bcd4fffeeba2f13c; no voters:
I20260812 21:12:43.355297 17666 leader_election.cc:288] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:43.355331 17676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:43.355474 17676 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [term 1 LEADER]: Becoming Leader. State: Replica: c3b5223b19814cf2bcd4fffeeba2f13c, State: Running, Role: LEADER
I20260812 21:12:43.355528 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:43.355571 17666 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:43.355638 17676 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } }
I20260812 21:12:43.356024 17676 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b5223b19814cf2bcd4fffeeba2f13c. Latest consensus state: current_term: 1 leader_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } } }
I20260812 21:12:43.356109 17676 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:43.356078 17677 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b5223b19814cf2bcd4fffeeba2f13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45869 } } }
I20260812 21:12:43.356237 17677 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:43.356989 17692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:43.358232 17692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:43.358425 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:43.357313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42435 (local address 127.0.0.1:45869)
0812 21:12:43.357363 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:43.357365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:43.357377 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:12:43.357480 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:43.357481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:43.357487 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:43.357539 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:43.358161 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:43.358162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:43.358163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:43.358167 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:43.358182 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:43.358187 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:43.358239 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:43.358349 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 21:12:43.358363 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:12:43.359975 17692 catalog_manager.cc:1347] Generated new cluster ID: 55474b7dd77f46f5a6e23e7062831bf2
I20260812 21:12:43.360031 17692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:43.368734 17692 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:43.368899 17692 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:43.376883 17692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b5223b19814cf2bcd4fffeeba2f13c: Generated new TSK 0
I20260812 21:12:43.377059 17692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:43.383517 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17547
2026-08-12T21:12:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:43 dist-test-slave-lkx2 krb5kdc[17720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:43 dist-test-slave-lkx2 krb5kdc[17720](info): set up 2 sockets
Aug 12 21:12:43 dist-test-slave-lkx2 krb5kdc[17720](info): commencing operation
krb5kdc: starting...
W20260812 21:12:44.584375 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:44 dist-test-slave-lkx2 krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:44.710835 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:44.794000 17739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:44.794209 17739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:44.794232 17739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:44.795321 17739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:44.795353 17739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:44.795367 17739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:44.795377 17739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:44.795388 17739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:44.796835 17739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:44.797039 17739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:44.797212 17739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:44.801910 17749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:44.801919 17754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:44.801990 17750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:44.802047 17739 server_base.cc:1034] running on GCE node
I20260812 21:12:44.802412 17739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:44.802683 17739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:44.803824 17739 hybrid_clock.cc:648] HybridClock initialized: now 1786569164803805 us; error 33 us; skew 500 ppm
Aug 12 21:12:44 dist-test-slave-lkx2 krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:44.806180 17739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:44.806587 17739 webserver.cc:460] Webserver started at http://127.0.0.1:36171/ using document root <none> and password file <none>
I20260812 21:12:44.806772 17739 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:44.806813 17739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:44.806905 17739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:44.807937 17739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "f212720bfb664ab2892da863e960d07b"
format_stamp: "Formatted at 2026-08-12 21:12:44 on dist-test-slave-lkx2"
server_key: "6908e23e12e9d2ddb28d91429e51d94d"
server_key_iv: "7eb97a285a2868e7d5eae9d84b9fa0eb"
server_key_version: "encryptionkey@0"
I20260812 21:12:44.808231 17739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "f212720bfb664ab2892da863e960d07b"
format_stamp: "Formatted at 2026-08-12 21:12:44 on dist-test-slave-lkx2"
server_key: "6908e23e12e9d2ddb28d91429e51d94d"
server_key_iv: "7eb97a285a2868e7d5eae9d84b9fa0eb"
server_key_version: "encryptionkey@0"
I20260812 21:12:44.809530 17739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:44.810287 17765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:44.810463 17739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:44.810554 17739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "f212720bfb664ab2892da863e960d07b"
format_stamp: "Formatted at 2026-08-12 21:12:44 on dist-test-slave-lkx2"
server_key: "6908e23e12e9d2ddb28d91429e51d94d"
server_key_iv: "7eb97a285a2868e7d5eae9d84b9fa0eb"
server_key_version: "encryptionkey@0"
I20260812 21:12:44.810611 17739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:44.840225 17739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:44.842528 17739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:44.842650 17739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:44.846561 17739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33649
I20260812 21:12:44.846596 17854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33649 every 8 connection(s)
I20260812 21:12:44.847031 17739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:44.847910 17855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:44.850313 17855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b: Bootstrap starting.
I20260812 21:12:44.850893 17855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:44.851161 17855 log.cc:824] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:44.851867 17855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b: No bootstrap required, opened a new log
I20260812 21:12:44.853572 17855 raft_consensus.cc:348] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f212720bfb664ab2892da863e960d07b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33649 } }
I20260812 21:12:44.853717 17855 raft_consensus.cc:374] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:44.853742 17855 raft_consensus.cc:729] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f212720bfb664ab2892da863e960d07b, State: Initialized, Role: FOLLOWER
I20260812 21:12:44.853891 17855 consensus_queue.cc:260] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f212720bfb664ab2892da863e960d07b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33649 } }
I20260812 21:12:44.853971 17855 raft_consensus.cc:388] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:44.854007 17855 raft_consensus.cc:482] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:44.854056 17855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:44.854595 17855 raft_consensus.cc:504] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f212720bfb664ab2892da863e960d07b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33649 } }
I20260812 21:12:44.854741 17855 leader_election.cc:302] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f212720bfb664ab2892da863e960d07b; no voters:
I20260812 21:12:44.854943 17855 leader_election.cc:288] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:44.854992 17858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:44.855168 17858 raft_consensus.cc:686] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [term 1 LEADER]: Becoming Leader. State: Replica: f212720bfb664ab2892da863e960d07b, State: Running, Role: LEADER
I20260812 21:12:44.855229 17855 sys_catalog.cc:564] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:44.855266 17858 consensus_queue.cc:237] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f212720bfb664ab2892da863e960d07b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33649 } }
I20260812 21:12:44.855659 17862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f212720bfb664ab2892da863e960d07b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f212720bfb664ab2892da863e960d07b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33649 } } }
I20260812 21:12:44.855756 17862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:44.855669 17864 sys_catalog.cc:455] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f212720bfb664ab2892da863e960d07b. Latest consensus state: current_term: 1 leader_uuid: "f212720bfb664ab2892da863e960d07b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f212720bfb664ab2892da863e960d07b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33649 } } }
I20260812 21:12:44.855840 17864 sys_catalog.cc:458] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:44.856022 17866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:44.856618 17866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:44.856827 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17739
I20260812 21:12:44.856969 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:44.857156 12094 external_mini_cluster.cc:1428] Setting key 4322c81438c3f8f798a7bb68b47bf367
I20260812 21:12:44.858487 17866 catalog_manager.cc:1347] Generated new cluster ID: ac2e57de39fa4f0fb6f3c2d77941b8a5
I20260812 21:12:44.858541 17866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:44 dist-test-slave-lkx2 krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:12:44.869290 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:44.857784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55811 (local address 127.0.0.1:33649)
0812 21:12:44.857959 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:44.857969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:44.858001 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:12:44.858117 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:44.858124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:44.858181 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:44.858323 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:44.859114 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:44.860580 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:44.861650 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:44.862347 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:44.862443 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:44.865225 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:44.865230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:44.865230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:44.865246 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:44.867204 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:44.867622 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:44.867626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:44.867626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:44.867734 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:44.867924 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:44.867925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:44.867926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:44.868243 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:44.868439 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:44.868895 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 21:12:44.869041 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260812 21:12:44.870391 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:44.878872 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:44.871810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33805 (local address 127.0.0.1:33649)
0812 21:12:44.871848 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:44.871849 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:44.872117 (+ 268us) server_negotiation.cc:407] Connection header received
0812 21:12:44.872149 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:44.872151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:44.872162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:44.872225 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:44.872834 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:44.873608 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:44.874554 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:44.874887 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:44.874952 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:44.876227 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:44.876228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:44.876228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:44.876235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:44.877727 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:44.878097 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:44.878098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:44.878098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:44.878165 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:44.878355 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:44.878356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:44.878356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:44.878491 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:44.878585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:44.878710 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 21:12:44.878788 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:12:44.879122 17866 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:44.879745 17866 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:44.884771 17866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f212720bfb664ab2892da863e960d07b: Generated new TSK 0
I20260812 21:12:44.885015 17866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:44.892324 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17739
2026-08-12T21:12:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:44 dist-test-slave-lkx2 krb5kdc[17899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:44 dist-test-slave-lkx2 krb5kdc[17899](info): set up 2 sockets
Aug 12 21:12:44 dist-test-slave-lkx2 krb5kdc[17899](info): commencing operation
krb5kdc: starting...
W20260812 21:12:46.095996 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:46 dist-test-slave-lkx2 krb5kdc[17899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:46.219349 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42047
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:46.301954 17931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:46.302176 17931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:46.302203 17931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:46.303314 17931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:46.303347 17931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:46.303359 17931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:46.303370 17931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:46.303381 17931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:46.304848 17931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:46.305048 17931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:46.305227 17931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:46.309999 17944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:46.310052 17931 server_base.cc:1034] running on GCE node
W20260812 21:12:46.310045 17947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:46.310005 17938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:46.310441 17931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:46.310712 17931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:46.311849 17931 hybrid_clock.cc:648] HybridClock initialized: now 1786569166311843 us; error 31 us; skew 500 ppm
Aug 12 21:12:46 dist-test-slave-lkx2 krb5kdc[17899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:46.317674 17931 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:46.318147 17931 webserver.cc:460] Webserver started at http://127.0.0.1:34741/ using document root <none> and password file <none>
I20260812 21:12:46.318351 17931 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:46.318400 17931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:46.318526 17931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:46.319571 17931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "4a7b5c3f614a4994a9baa7832a183aa9"
format_stamp: "Formatted at 2026-08-12 21:12:46 on dist-test-slave-lkx2"
server_key: "0eabef2a01df0941a0e97af1383dbfec"
server_key_iv: "a9706f57e0b54ab7286bd33b5253f8c1"
server_key_version: "encryptionkey@0"
I20260812 21:12:46.319907 17931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "4a7b5c3f614a4994a9baa7832a183aa9"
format_stamp: "Formatted at 2026-08-12 21:12:46 on dist-test-slave-lkx2"
server_key: "0eabef2a01df0941a0e97af1383dbfec"
server_key_iv: "a9706f57e0b54ab7286bd33b5253f8c1"
server_key_version: "encryptionkey@0"
I20260812 21:12:46.321337 17931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:12:46.322108 17958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:46.322279 17931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:46.322366 17931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "4a7b5c3f614a4994a9baa7832a183aa9"
format_stamp: "Formatted at 2026-08-12 21:12:46 on dist-test-slave-lkx2"
server_key: "0eabef2a01df0941a0e97af1383dbfec"
server_key_iv: "a9706f57e0b54ab7286bd33b5253f8c1"
server_key_version: "encryptionkey@0"
I20260812 21:12:46.322444 17931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:46.349045 17931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:46.350968 17931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:46.351085 17931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:46.355343 17931 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42047
I20260812 21:12:46.355393 18045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42047 every 8 connection(s)
I20260812 21:12:46.355759 17931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:46.356680 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:46.359091 18047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9: Bootstrap starting.
I20260812 21:12:46.359694 18047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:46.359962 18047 log.cc:824] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:46.360680 18047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9: No bootstrap required, opened a new log
I20260812 21:12:46.362342 18047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 21:12:46.362491 18047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:46.362514 18047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a7b5c3f614a4994a9baa7832a183aa9, State: Initialized, Role: FOLLOWER
I20260812 21:12:46.362655 18047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 21:12:46.362731 18047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:46.362768 18047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:46.362818 18047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:46.363380 18047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 21:12:46.363477 18047 leader_election.cc:302] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a7b5c3f614a4994a9baa7832a183aa9; no voters:
I20260812 21:12:46.363659 18047 leader_election.cc:288] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:46.363703 18055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:46.363864 18055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [term 1 LEADER]: Becoming Leader. State: Replica: 4a7b5c3f614a4994a9baa7832a183aa9, State: Running, Role: LEADER
I20260812 21:12:46.363931 18047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:46.363965 18055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } }
I20260812 21:12:46.364391 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a7b5c3f614a4994a9baa7832a183aa9. Latest consensus state: current_term: 1 leader_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } } }
I20260812 21:12:46.364537 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7b5c3f614a4994a9baa7832a183aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42047 } } }
I20260812 21:12:46.364523 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:46.364763 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:46.365010 18073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:46.365146 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 17931
I20260812 21:12:46.365269 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:46.365471 12094 external_mini_cluster.cc:1428] Setting key 2481c5002bf5236b8ac350db121795c6
I20260812 21:12:46.365598 18073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:46.367305 18073 catalog_manager.cc:1347] Generated new cluster ID: 724a42d31b944add89a85c91285367ef
I20260812 21:12:46.367357 18073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:46.370679 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:46.366166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40075 (local address 127.0.0.1:42047)
0812 21:12:46.366335 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:46.366345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:46.366381 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:12:46.366499 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:46.366507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:46.366699 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:46.366851 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:46.367079 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:46.368045 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:46.368936 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:46.369342 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:46.369438 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:46.370061 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:46.370065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:46.370065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:46.370232 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:46.370268 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:46.370278 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:46.370336 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:46.370407 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:12:46.370468 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260812 21:12:46.371572 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:46.373844 18073 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:46.374462 18073 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:46.377031 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:46.373780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32861 (local address 127.0.0.1:42047)
0812 21:12:46.373812 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:46.373814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:46.373858 (+ 44us) server_negotiation.cc:407] Connection header received
0812 21:12:46.373949 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:46.373950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:46.373990 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:46.374055 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:46.374199 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:46.374851 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:46.375814 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:46.376096 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:46.376148 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:46.376801 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:46.376803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:46.376803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:46.376808 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:46.376822 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:46.376826 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:46.376867 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:46.376926 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 21:12:46.376962 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:12:46.385603 18073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a7b5c3f614a4994a9baa7832a183aa9: Generated new TSK 0
I20260812 21:12:46.385836 18073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:46.390312 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 17931
2026-08-12T21:12:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:46 dist-test-slave-lkx2 krb5kdc[18105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:46 dist-test-slave-lkx2 krb5kdc[18105](info): set up 2 sockets
Aug 12 21:12:46 dist-test-slave-lkx2 krb5kdc[18105](info): commencing operation
krb5kdc: starting...
W20260812 21:12:47.629119 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:47 dist-test-slave-lkx2 krb5kdc[18105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:47.756076 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:47.841653 18125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:47.841879 18125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:47.841912 18125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:47.843112 18125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:47.843158 18125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:47.843178 18125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:47.843194 18125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:47.843209 18125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:47.844758 18125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:47.845005 18125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:47.845230 18125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:47.850184 18134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:47.850204 18136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:47.850283 18125 server_base.cc:1034] running on GCE node
W20260812 21:12:47.850184 18138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:47.850668 18125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:47.851002 18125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:47.852154 18125 hybrid_clock.cc:648] HybridClock initialized: now 1786569167852133 us; error 38 us; skew 500 ppm
Aug 12 21:12:47 dist-test-slave-lkx2 krb5kdc[18105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:47.854671 18125 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:47.855145 18125 webserver.cc:460] Webserver started at http://127.0.0.1:40531/ using document root <none> and password file <none>
I20260812 21:12:47.855371 18125 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:47.855422 18125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:47.855549 18125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:47.856644 18125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "4cb3ac6fe4e3467dba43c4f1d3948195"
format_stamp: "Formatted at 2026-08-12 21:12:47 on dist-test-slave-lkx2"
server_key: "0220943f648cf0284d06f671faead0c4"
server_key_iv: "d788daae52783a33fa53857c96608aed"
server_key_version: "encryptionkey@0"
I20260812 21:12:47.856974 18125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "4cb3ac6fe4e3467dba43c4f1d3948195"
format_stamp: "Formatted at 2026-08-12 21:12:47 on dist-test-slave-lkx2"
server_key: "0220943f648cf0284d06f671faead0c4"
server_key_iv: "d788daae52783a33fa53857c96608aed"
server_key_version: "encryptionkey@0"
I20260812 21:12:47.858247 18125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:47.859023 18154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:47.859202 18125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:47.859290 18125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "4cb3ac6fe4e3467dba43c4f1d3948195"
format_stamp: "Formatted at 2026-08-12 21:12:47 on dist-test-slave-lkx2"
server_key: "0220943f648cf0284d06f671faead0c4"
server_key_iv: "d788daae52783a33fa53857c96608aed"
server_key_version: "encryptionkey@0"
I20260812 21:12:47.859352 18125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:47.862749 18125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:47.864660 18125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:47.864794 18125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:47.868731 18125 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46037
I20260812 21:12:47.868763 18244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46037 every 8 connection(s)
I20260812 21:12:47.869095 18125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:47.870071 18247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:47.871915 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18125
I20260812 21:12:47.872025 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:47.872210 12094 external_mini_cluster.cc:1428] Setting key 280abe154ea6da02672cdc5bd0c0faee
I20260812 21:12:47.872655 18247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195: Bootstrap starting.
I20260812 21:12:47.873391 18247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:47.873724 18247 log.cc:824] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:47.874648 18247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195: No bootstrap required, opened a new log
I20260812 21:12:47.874866 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:47.872730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50485 (local address 127.0.0.1:46037)
0812 21:12:47.872940 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:47.872950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:47.872987 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:12:47.873581 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:47.873591 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:47.873627 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:47.873795 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:47.874439 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:47.874443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:47.874444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:47.874448 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:47.874494 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:47.874508 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:47.874602 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:47.874633 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 21:12:47.874683 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260812 21:12:47.875854 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:47.876911 18247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260812 21:12:47.877058 18247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:47.877091 18247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb3ac6fe4e3467dba43c4f1d3948195, State: Initialized, Role: FOLLOWER
I20260812 21:12:47.877228 18247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260812 21:12:47.877336 18247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:47.877377 18247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:47.877426 18247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:47.878104 18247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260812 21:12:47.878230 18247 leader_election.cc:302] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cb3ac6fe4e3467dba43c4f1d3948195; no voters:
I20260812 21:12:47.878309 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:47.877258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36779 (local address 127.0.0.1:46037)
0812 21:12:47.877291 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:47.877293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:47.877393 (+ 100us) server_negotiation.cc:407] Connection header received
0812 21:12:47.877462 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:47.877463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:47.877468 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:47.877516 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:47.878064 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:47.878065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:47.878065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:47.878069 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:47.878083 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:47.878086 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:47.878131 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:47.878189 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 21:12:47.878216 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:12:47.878404 18247 leader_election.cc:288] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:47.878521 18262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:47.878654 18247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:47.878830 18262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [term 1 LEADER]: Becoming Leader. State: Replica: 4cb3ac6fe4e3467dba43c4f1d3948195, State: Running, Role: LEADER
I20260812 21:12:47.878937 18262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260812 21:12:47.879326 18264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } } }
I20260812 21:12:47.879416 18264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:47.879336 18265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cb3ac6fe4e3467dba43c4f1d3948195. Latest consensus state: current_term: 1 leader_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb3ac6fe4e3467dba43c4f1d3948195" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } } }
I20260812 21:12:47.879619 18265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:47.879632 18276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:47.880111 18276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:47.881675 18276 catalog_manager.cc:1347] Generated new cluster ID: d45ce92801414a1e972b6f989ff1357e
I20260812 21:12:47.881716 18276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:47.888725 18276 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:47.888875 18276 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:47.891790 18276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cb3ac6fe4e3467dba43c4f1d3948195: Generated new TSK 0
I20260812 21:12:47.891947 18276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:47.896881 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18125
2026-08-12T21:12:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:47 dist-test-slave-lkx2 krb5kdc[18295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:47 dist-test-slave-lkx2 krb5kdc[18295](info): set up 2 sockets
Aug 12 21:12:47 dist-test-slave-lkx2 krb5kdc[18295](info): commencing operation
krb5kdc: starting...
W20260812 21:12:49.127664 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:49 dist-test-slave-lkx2 krb5kdc[18295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:49.251019 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:49.332754 18314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:49.332973 18314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:49.333003 18314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:49.334187 18314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:49.334229 18314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:49.334247 18314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:49.334275 18314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:49.334290 18314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:49.335777 18314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:49.336024 18314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:49.336238 18314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:49.341071 18329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:49.341100 18322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:49.341044 18326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:49.341269 18314 server_base.cc:1034] running on GCE node
I20260812 21:12:49.341571 18314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:49.341859 18314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:49.342996 18314 hybrid_clock.cc:648] HybridClock initialized: now 1786569169342983 us; error 30 us; skew 500 ppm
Aug 12 21:12:49 dist-test-slave-lkx2 krb5kdc[18295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:49.345667 18314 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:49.346143 18314 webserver.cc:460] Webserver started at http://127.0.0.1:43353/ using document root <none> and password file <none>
I20260812 21:12:49.346344 18314 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:49.346388 18314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:49.346490 18314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:49.347548 18314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "4956af37762f49c99e98d90413050e3e"
format_stamp: "Formatted at 2026-08-12 21:12:49 on dist-test-slave-lkx2"
server_key: "cf23ee1da833df5cb190dd9044ea288c"
server_key_iv: "4b1c29a072890b4f36c946995405ccb1"
server_key_version: "encryptionkey@0"
I20260812 21:12:49.347878 18314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "4956af37762f49c99e98d90413050e3e"
format_stamp: "Formatted at 2026-08-12 21:12:49 on dist-test-slave-lkx2"
server_key: "cf23ee1da833df5cb190dd9044ea288c"
server_key_iv: "4b1c29a072890b4f36c946995405ccb1"
server_key_version: "encryptionkey@0"
I20260812 21:12:49.349323 18314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:49.350091 18348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:49.350287 18314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:49.350374 18314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "4956af37762f49c99e98d90413050e3e"
format_stamp: "Formatted at 2026-08-12 21:12:49 on dist-test-slave-lkx2"
server_key: "cf23ee1da833df5cb190dd9044ea288c"
server_key_iv: "4b1c29a072890b4f36c946995405ccb1"
server_key_version: "encryptionkey@0"
I20260812 21:12:49.350433 18314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:49.385169 18314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:49.387629 18314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:49.387755 18314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:49.391770 18314 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46059
I20260812 21:12:49.391839 18448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46059 every 8 connection(s)
I20260812 21:12:49.392161 18314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:49.393074 18449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:49.395427 18449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e: Bootstrap starting.
I20260812 21:12:49.395996 18449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:49.396239 18449 log.cc:824] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:49.396790 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18314
I20260812 21:12:49.396906 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:49.397092 12094 external_mini_cluster.cc:1428] Setting key e509c4378219f5769bbaf7ba6ec002a6
I20260812 21:12:49.397105 18449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e: No bootstrap required, opened a new log
I20260812 21:12:49.398934 18449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956af37762f49c99e98d90413050e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260812 21:12:49.399073 18449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:49.399104 18449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4956af37762f49c99e98d90413050e3e, State: Initialized, Role: FOLLOWER
I20260812 21:12:49.399276 18449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956af37762f49c99e98d90413050e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260812 21:12:49.399369 18449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:49.399400 18449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:49.399441 18449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:49.400456 18449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956af37762f49c99e98d90413050e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260812 21:12:49.400556 18449 leader_election.cc:302] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4956af37762f49c99e98d90413050e3e; no voters:
I20260812 21:12:49.400727 18449 leader_election.cc:288] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:49.400761 18457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:49.400870 18457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [term 1 LEADER]: Becoming Leader. State: Replica: 4956af37762f49c99e98d90413050e3e, State: Running, Role: LEADER
I20260812 21:12:49.400998 18449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:49.401016 18457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956af37762f49c99e98d90413050e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260812 21:12:49.401381 18457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4956af37762f49c99e98d90413050e3e. Latest consensus state: current_term: 1 leader_uuid: "4956af37762f49c99e98d90413050e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956af37762f49c99e98d90413050e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260812 21:12:49.401459 18457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:49.401729 18463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:49.402343 18463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:49.402369 18459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4956af37762f49c99e98d90413050e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956af37762f49c99e98d90413050e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260812 21:12:49.402500 18459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:49.404023 18463 catalog_manager.cc:1347] Generated new cluster ID: ff3899fdbfb54f5a97f7fac4f2c49220
I20260812 21:12:49.404066 18463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:49 dist-test-slave-lkx2 krb5kdc[18295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:12:49.410248 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:49.397805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:48533 (local address 127.0.0.1:46059)
0812 21:12:49.397982 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:49.397990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:49.398017 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:12:49.398929 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:49.398937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:49.398994 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:49.399127 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:49.399936 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:49.401127 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:49.402163 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:49.402770 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:49.402883 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:49.405980 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:49.405985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:49.405985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:49.405998 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:49.407954 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:49.408509 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:49.408513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:49.408513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:49.408673 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:49.408967 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:49.408968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:49.408968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:49.409284 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:49.409453 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:49.409935 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 21:12:49.410045 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260812 21:12:49.411646 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:49.421036 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:49.413252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:50635 (local address 127.0.0.1:46059)
0812 21:12:49.413293 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:49.413294 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:49.413439 (+ 145us) server_negotiation.cc:407] Connection header received
0812 21:12:49.413540 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:49.413541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:49.413554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:49.413586 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:49.414347 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:49.415081 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:49.416127 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:49.416474 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:49.416521 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:49.418033 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:49.418035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:49.418035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:49.418042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:49.419525 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:49.420004 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:49.420005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:49.420005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:49.420063 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:49.420325 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:49.420326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:49.420326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:49.420499 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:49.420585 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:49.420861 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 21:12:49.420933 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:12:49.423606 18463 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:49.424332 18463 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:49.437597 18463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4956af37762f49c99e98d90413050e3e: Generated new TSK 0
I20260812 21:12:49.437858 18463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:49.442803 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18314
W20260812 21:12:49.457650 18292 connection.cc:535] client connection to 127.0.0.1:46059 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46059): Connection reset by peer (error 104)
2026-08-12T21:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:49 dist-test-slave-lkx2 krb5kdc[18512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:49 dist-test-slave-lkx2 krb5kdc[18512](info): set up 2 sockets
Aug 12 21:12:49 dist-test-slave-lkx2 krb5kdc[18512](info): commencing operation
krb5kdc: starting...
W20260812 21:12:50.646806 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:50 dist-test-slave-lkx2 krb5kdc[18512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:50.770999 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:50.855391 18530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:50.855614 18530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:50.855638 18530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:50.856807 18530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:50.856845 18530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:50.856858 18530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:50.856868 18530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:50.856879 18530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:50.858390 18530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:50.858608 18530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:50.858789 18530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:50.863719 18537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:50.863672 18540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:50.863672 18538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:50.863965 18530 server_base.cc:1034] running on GCE node
I20260812 21:12:50.864202 18530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:50.864534 18530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:50.865680 18530 hybrid_clock.cc:648] HybridClock initialized: now 1786569170865658 us; error 32 us; skew 500 ppm
Aug 12 21:12:50 dist-test-slave-lkx2 krb5kdc[18512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:50.868202 18530 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:50.868692 18530 webserver.cc:460] Webserver started at http://127.0.0.1:34111/ using document root <none> and password file <none>
I20260812 21:12:50.868913 18530 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:50.868957 18530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:50.869081 18530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:50.870175 18530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "110fd326be354ec4a11f5090d3109467"
format_stamp: "Formatted at 2026-08-12 21:12:50 on dist-test-slave-lkx2"
server_key: "9136f42b230c3fae3f035311958cb7b5"
server_key_iv: "2e08e40f6e86f76b1d4c8e97fccc3669"
server_key_version: "encryptionkey@0"
I20260812 21:12:50.870513 18530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "110fd326be354ec4a11f5090d3109467"
format_stamp: "Formatted at 2026-08-12 21:12:50 on dist-test-slave-lkx2"
server_key: "9136f42b230c3fae3f035311958cb7b5"
server_key_iv: "2e08e40f6e86f76b1d4c8e97fccc3669"
server_key_version: "encryptionkey@0"
I20260812 21:12:50.871848 18530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:50.872628 18554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:50.872777 18530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:50.872874 18530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "110fd326be354ec4a11f5090d3109467"
format_stamp: "Formatted at 2026-08-12 21:12:50 on dist-test-slave-lkx2"
server_key: "9136f42b230c3fae3f035311958cb7b5"
server_key_iv: "2e08e40f6e86f76b1d4c8e97fccc3669"
server_key_version: "encryptionkey@0"
I20260812 21:12:50.872942 18530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:50.885000 18530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:50.886858 18530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:50.886996 18530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:50.891042 18530 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36857
I20260812 21:12:50.891100 18654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36857 every 8 connection(s)
I20260812 21:12:50.891436 18530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:50.892302 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:50.894639 18655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467: Bootstrap starting.
I20260812 21:12:50.895181 18655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:50.895501 18655 log.cc:824] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:50.896291 18655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467: No bootstrap required, opened a new log
I20260812 21:12:50.896849 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18530
I20260812 21:12:50.896961 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:50.897123 12094 external_mini_cluster.cc:1428] Setting key bb1cde01092615841529793bbfa69d9f
I20260812 21:12:50.898458 18655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110fd326be354ec4a11f5090d3109467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260812 21:12:50.898612 18655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:50.898694 18655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110fd326be354ec4a11f5090d3109467, State: Initialized, Role: FOLLOWER
I20260812 21:12:50.898864 18655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110fd326be354ec4a11f5090d3109467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260812 21:12:50.898945 18655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:50.898980 18655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:50.899040 18655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:50.899811 18655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110fd326be354ec4a11f5090d3109467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260812 21:12:50.899927 18655 leader_election.cc:302] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 110fd326be354ec4a11f5090d3109467; no voters:
I20260812 21:12:50.900095 18655 leader_election.cc:288] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:50.900135 18662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:50.900285 18662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [term 1 LEADER]: Becoming Leader. State: Replica: 110fd326be354ec4a11f5090d3109467, State: Running, Role: LEADER
I20260812 21:12:50.900364 18655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:50.900450 18662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110fd326be354ec4a11f5090d3109467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260812 21:12:50.901779 18663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "110fd326be354ec4a11f5090d3109467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110fd326be354ec4a11f5090d3109467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } } }
I20260812 21:12:50.901885 18663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:50.902084 18664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 110fd326be354ec4a11f5090d3109467. Latest consensus state: current_term: 1 leader_uuid: "110fd326be354ec4a11f5090d3109467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110fd326be354ec4a11f5090d3109467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } } }
I20260812 21:12:50.902329 18664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467 [sys.catalog]: This master's current role is: LEADER
W20260812 21:12:50.902465 18687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:12:50.902509 18687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:12:50.902562 18688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:50.902770 18688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:50.903990 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:50.897619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:49055 (local address 127.0.0.1:36857)
0812 21:12:50.897821 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:50.897829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:50.897858 (+ 29us) server_negotiation.cc:407] Connection header received
0812 21:12:50.898601 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:50.898610 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:50.898683 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:50.898842 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:50.899082 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:50.900465 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:50.901596 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:50.902272 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:50.902380 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:50.903287 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:50.903291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:50.903291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:50.903495 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:50.903537 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:50.903553 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:50.903709 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:50.903804 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:12:50.903832 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260812 21:12:50.904633 18688 catalog_manager.cc:1347] Generated new cluster ID: 51ab5ce6d494417aad39f798fe968596
I20260812 21:12:50.904685 18688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:50.905023 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:50.909166 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:50.906313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:36793 (local address 127.0.0.1:36857)
0812 21:12:50.906415 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:50.906417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:50.906468 (+ 51us) server_negotiation.cc:407] Connection header received
0812 21:12:50.906543 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:50.906543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:50.906553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:50.906610 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:50.906743 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:50.907195 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:50.907889 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:50.908108 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:50.908170 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:50.908911 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:50.908913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:50.908913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:50.908918 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:50.908933 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:50.908937 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:50.908994 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:50.909075 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 21:12:50.909086 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 21:12:50.919674 18688 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:50.920244 18688 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:50.928685 18688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 110fd326be354ec4a11f5090d3109467: Generated new TSK 0
I20260812 21:12:50.928898 18688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:50.938356 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18530
W20260812 21:12:50.960275 18507 connection.cc:535] client connection to 127.0.0.1:36857 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36857): Connection reset by peer (error 104)
2026-08-12T21:12:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:51 dist-test-slave-lkx2 krb5kdc[18703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:51 dist-test-slave-lkx2 krb5kdc[18703](info): set up 2 sockets
Aug 12 21:12:51 dist-test-slave-lkx2 krb5kdc[18703](info): commencing operation
krb5kdc: starting...
W20260812 21:12:52.167956 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:52 dist-test-slave-lkx2 krb5kdc[18703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:52.290583 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:52.372442 18725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:52.372642 18725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:52.372664 18725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:52.373870 18725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:52.373912 18725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:52.373925 18725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:52.373939 18725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:52.373948 18725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:52.375509 18725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:52.375748 18725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:52.375974 18725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:52.380800 18741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:52.380828 18725 server_base.cc:1034] running on GCE node
W20260812 21:12:52.380801 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:52.380812 18740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:52.381268 18725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:52.381541 18725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:52.382717 18725 hybrid_clock.cc:648] HybridClock initialized: now 1786569172382698 us; error 35 us; skew 500 ppm
Aug 12 21:12:52 dist-test-slave-lkx2 krb5kdc[18703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:52.388651 18725 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:52.389127 18725 webserver.cc:460] Webserver started at http://127.0.0.1:43013/ using document root <none> and password file <none>
I20260812 21:12:52.389353 18725 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:52.389401 18725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:52.389529 18725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:52.390547 18725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "0bb817c488c9489fb6bcae9c741b29c2"
format_stamp: "Formatted at 2026-08-12 21:12:52 on dist-test-slave-lkx2"
server_key: "7dfe9bffbf8d267df628e80ad4b4b4fc"
server_key_iv: "abca2281772924ffbe4530f1840f087a"
server_key_version: "encryptionkey@0"
I20260812 21:12:52.390877 18725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "0bb817c488c9489fb6bcae9c741b29c2"
format_stamp: "Formatted at 2026-08-12 21:12:52 on dist-test-slave-lkx2"
server_key: "7dfe9bffbf8d267df628e80ad4b4b4fc"
server_key_iv: "abca2281772924ffbe4530f1840f087a"
server_key_version: "encryptionkey@0"
I20260812 21:12:52.392097 18725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:12:52.392813 18759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:52.392967 18725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:52.393056 18725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "0bb817c488c9489fb6bcae9c741b29c2"
format_stamp: "Formatted at 2026-08-12 21:12:52 on dist-test-slave-lkx2"
server_key: "7dfe9bffbf8d267df628e80ad4b4b4fc"
server_key_iv: "abca2281772924ffbe4530f1840f087a"
server_key_version: "encryptionkey@0"
I20260812 21:12:52.393121 18725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:52.396484 18725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:52.398571 18725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:52.398699 18725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:52.402575 18725 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39207
I20260812 21:12:52.402609 18857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39207 every 8 connection(s)
I20260812 21:12:52.402925 18725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:52.403964 18859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:52.405615 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18725
I20260812 21:12:52.405725 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:52.405892 12094 external_mini_cluster.cc:1428] Setting key 57d4b1d595a70c57dc02c220fe9e9ed6
I20260812 21:12:52.406569 18859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2: Bootstrap starting.
I20260812 21:12:52.407327 18859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:52.407706 18859 log.cc:824] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:52.408519 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:52.406394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:51887 (local address 127.0.0.1:39207)
0812 21:12:52.406587 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:52.406597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:52.406634 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:12:52.407237 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:52.407246 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:52.407283 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:52.407455 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:52.408034 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:52.408038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:52.408039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:52.408044 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:52.408103 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:52.408119 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:52.408218 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:52.408256 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 21:12:52.408297 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260812 21:12:52.408582 18859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2: No bootstrap required, opened a new log
I20260812 21:12:52.409539 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:52.410619 18859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb817c488c9489fb6bcae9c741b29c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260812 21:12:52.410753 18859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:52.410774 18859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb817c488c9489fb6bcae9c741b29c2, State: Initialized, Role: FOLLOWER
I20260812 21:12:52.410902 18859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb817c488c9489fb6bcae9c741b29c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260812 21:12:52.411027 18859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:52.411098 18859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:52.411149 18859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:52.411871 18859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb817c488c9489fb6bcae9c741b29c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260812 21:12:52.411983 18859 leader_election.cc:302] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bb817c488c9489fb6bcae9c741b29c2; no voters:
I20260812 21:12:52.412081 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:52.411025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:52383 (local address 127.0.0.1:39207)
0812 21:12:52.411066 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:52.411067 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:52.411172 (+ 105us) server_negotiation.cc:407] Connection header received
0812 21:12:52.411248 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:52.411249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:52.411255 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:52.411319 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:52.411860 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:52.411861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:52.411862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:52.411867 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:52.411881 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:52.411886 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:52.411945 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:52.412014 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 21:12:52.412028 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:12:52.412150 18859 leader_election.cc:288] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:52.412302 18874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:52.412379 18859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:52.412534 18874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [term 1 LEADER]: Becoming Leader. State: Replica: 0bb817c488c9489fb6bcae9c741b29c2, State: Running, Role: LEADER
I20260812 21:12:52.412693 18874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb817c488c9489fb6bcae9c741b29c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } }
I20260812 21:12:52.413147 18876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bb817c488c9489fb6bcae9c741b29c2. Latest consensus state: current_term: 1 leader_uuid: "0bb817c488c9489fb6bcae9c741b29c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb817c488c9489fb6bcae9c741b29c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } } }
I20260812 21:12:52.413298 18876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:52.413168 18873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bb817c488c9489fb6bcae9c741b29c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb817c488c9489fb6bcae9c741b29c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39207 } } }
I20260812 21:12:52.413518 18873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:52.413535 18885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:52.414032 18885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:52.415558 18885 catalog_manager.cc:1347] Generated new cluster ID: ac228eb7b624432ab53ca9aaa09ea0e3
I20260812 21:12:52.415611 18885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:52.429379 18885 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:52.429580 18885 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:52.432621 18885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bb817c488c9489fb6bcae9c741b29c2: Generated new TSK 0
I20260812 21:12:52.432827 18885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:52.435410 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18725
2026-08-12T21:12:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:52 dist-test-slave-lkx2 krb5kdc[18905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:52 dist-test-slave-lkx2 krb5kdc[18905](info): set up 2 sockets
Aug 12 21:12:52 dist-test-slave-lkx2 krb5kdc[18905](info): commencing operation
krb5kdc: starting...
W20260812 21:12:53.636068 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:53 dist-test-slave-lkx2 krb5kdc[18905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:53.760730 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:53.845784 18929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:53.846000 18929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:53.846024 18929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:53.847196 18929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:53.847239 18929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:53.847251 18929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:53.847275 18929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:53.847297 18929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:53.848805 18929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:53.849045 18929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:53.849261 18929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:53.854079 18937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:53.854197 18940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:53.854102 18929 server_base.cc:1034] running on GCE node
W20260812 21:12:53.854086 18942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:53.854519 18929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:53.854816 18929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:53.855960 18929 hybrid_clock.cc:648] HybridClock initialized: now 1786569173855946 us; error 29 us; skew 500 ppm
Aug 12 21:12:53 dist-test-slave-lkx2 krb5kdc[18905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569173, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:53.858466 18929 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:53.858868 18929 webserver.cc:460] Webserver started at http://127.0.0.1:33575/ using document root <none> and password file <none>
I20260812 21:12:53.859047 18929 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:53.859084 18929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:53.859175 18929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:53.860216 18929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "383e6cd1914c45959cc126f302ff8244"
format_stamp: "Formatted at 2026-08-12 21:12:53 on dist-test-slave-lkx2"
server_key: "6234d8ce8c374cc0b5048733b871aeb5"
server_key_iv: "ed4e4537ad765ed80233cd58dc5e15e9"
server_key_version: "encryptionkey@0"
I20260812 21:12:53.860598 18929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "383e6cd1914c45959cc126f302ff8244"
format_stamp: "Formatted at 2026-08-12 21:12:53 on dist-test-slave-lkx2"
server_key: "6234d8ce8c374cc0b5048733b871aeb5"
server_key_iv: "ed4e4537ad765ed80233cd58dc5e15e9"
server_key_version: "encryptionkey@0"
I20260812 21:12:53.861876 18929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:53.862535 18951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:53.862691 18929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:53.862813 18929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "383e6cd1914c45959cc126f302ff8244"
format_stamp: "Formatted at 2026-08-12 21:12:53 on dist-test-slave-lkx2"
server_key: "6234d8ce8c374cc0b5048733b871aeb5"
server_key_iv: "ed4e4537ad765ed80233cd58dc5e15e9"
server_key_version: "encryptionkey@0"
I20260812 21:12:53.862874 18929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:53.881494 18929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:53.883490 18929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:53.883610 18929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:53.887549 18929 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34759
I20260812 21:12:53.887593 19037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34759 every 8 connection(s)
I20260812 21:12:53.887939 18929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:53.888831 19038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:53.891170 19038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244: Bootstrap starting.
I20260812 21:12:53.891736 19038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:53.892026 19038 log.cc:824] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:53.892727 19038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244: No bootstrap required, opened a new log
I20260812 21:12:53.894335 19038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383e6cd1914c45959cc126f302ff8244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34759 } }
I20260812 21:12:53.894474 19038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:53.894510 19038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383e6cd1914c45959cc126f302ff8244, State: Initialized, Role: FOLLOWER
I20260812 21:12:53.894665 19038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383e6cd1914c45959cc126f302ff8244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34759 } }
I20260812 21:12:53.894749 19038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:53.894793 19038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:53.894843 19038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:53.895359 19038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383e6cd1914c45959cc126f302ff8244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34759 } }
I20260812 21:12:53.895455 19038 leader_election.cc:302] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 383e6cd1914c45959cc126f302ff8244; no voters:
I20260812 21:12:53.895612 19038 leader_election.cc:288] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:53.895668 19043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:53.895850 19043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [term 1 LEADER]: Becoming Leader. State: Replica: 383e6cd1914c45959cc126f302ff8244, State: Running, Role: LEADER
I20260812 21:12:53.895869 19038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:53.895996 19043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383e6cd1914c45959cc126f302ff8244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34759 } }
I20260812 21:12:53.896437 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 18929
I20260812 21:12:53.896569 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:53.896783 12094 external_mini_cluster.cc:1428] Setting key 481ef2e4a61d66ea9f2ead19925b849f
I20260812 21:12:53.897260 19045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "383e6cd1914c45959cc126f302ff8244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383e6cd1914c45959cc126f302ff8244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34759 } } }
I20260812 21:12:53.897560 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 383e6cd1914c45959cc126f302ff8244. Latest consensus state: current_term: 1 leader_uuid: "383e6cd1914c45959cc126f302ff8244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383e6cd1914c45959cc126f302ff8244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34759 } } }
I20260812 21:12:53.897363 19045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:53.897815 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244 [sys.catalog]: This master's current role is: LEADER
W20260812 21:12:53.898223 19073 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:12:53.898279 19073 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:12:53.898339 19078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:53.898517 19078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:53.900743 19078 catalog_manager.cc:1347] Generated new cluster ID: 836ecfefc4d04c08b2bc72f69a01df2d
I20260812 21:12:53.900807 19078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:53 dist-test-slave-lkx2 krb5kdc[18905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:12:53.907951 19078 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:53.908609 19078 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:53.909157 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:53.897305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:51339 (local address 127.0.0.1:34759)
0812 21:12:53.897515 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:53.897525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:53.897672 (+ 147us) server_negotiation.cc:407] Connection header received
0812 21:12:53.897861 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:53.897868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:53.897949 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:53.898111 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:53.898909 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:53.900265 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:53.901316 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:53.901916 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:53.902040 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:53.904812 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:53.904816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:53.904816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:53.904828 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:53.907379 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:53.907798 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:53.907801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:53.907801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:53.907926 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:53.908189 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:53.908190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:53.908190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:53.908517 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:53.908711 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:53.908804 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 21:12:53.908920 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260812 21:12:53.910151 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:53.912207 19078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 383e6cd1914c45959cc126f302ff8244: Generated new TSK 0
I20260812 21:12:53.912411 19078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:53.917753 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:53.911700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:33627 (local address 127.0.0.1:34759)
0812 21:12:53.911739 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:53.911741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:53.911958 (+ 217us) server_negotiation.cc:407] Connection header received
0812 21:12:53.912223 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:53.912224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:53.912233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:53.912280 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:53.912861 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:53.913378 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:53.914064 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:53.914266 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:53.914326 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:53.915449 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:53.915450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:53.915450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:53.915457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:53.916719 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:53.917050 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:53.917051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:53.917051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:53.917102 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:53.917298 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:53.917299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:53.917299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:53.917397 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:53.917504 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:53.917611 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 21:12:53.917670 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:12:53.919504 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 18929
W20260812 21:12:53.939471 18901 connection.cc:535] client connection to 127.0.0.1:34759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34759): Connection reset by peer (error 104)
2026-08-12T21:12:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:53 dist-test-slave-lkx2 krb5kdc[19093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:53 dist-test-slave-lkx2 krb5kdc[19093](info): set up 2 sockets
Aug 12 21:12:53 dist-test-slave-lkx2 krb5kdc[19093](info): commencing operation
krb5kdc: starting...
W20260812 21:12:55.131387 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:55 dist-test-slave-lkx2 krb5kdc[19093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:55.255160 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:55.337491 19116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:55.337707 19116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:55.337741 19116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:55.339121 19116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:55.339195 19116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:55.339219 19116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:55.339237 19116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:55.339252 19116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:55.340956 19116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:55.341228 19116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:55.341444 19116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:55.346719 19125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:55.346756 19123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:55.346792 19116 server_base.cc:1034] running on GCE node
W20260812 21:12:55.346756 19127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:55.347092 19116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:55.347409 19116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:55.348551 19116 hybrid_clock.cc:648] HybridClock initialized: now 1786569175348539 us; error 30 us; skew 500 ppm
Aug 12 21:12:55 dist-test-slave-lkx2 krb5kdc[19093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:55.351073 19116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:55.351523 19116 webserver.cc:460] Webserver started at http://127.0.0.1:35479/ using document root <none> and password file <none>
I20260812 21:12:55.351747 19116 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:55.351799 19116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:55.351924 19116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:55.353015 19116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "5b480ca38e234b10bd44ebcb19a6763b"
format_stamp: "Formatted at 2026-08-12 21:12:55 on dist-test-slave-lkx2"
server_key: "a10ec7bfe8ba13a7d8361f290f3e888f"
server_key_iv: "4937b871e023609c6bb527d141663a1d"
server_key_version: "encryptionkey@0"
I20260812 21:12:55.353340 19116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "5b480ca38e234b10bd44ebcb19a6763b"
format_stamp: "Formatted at 2026-08-12 21:12:55 on dist-test-slave-lkx2"
server_key: "a10ec7bfe8ba13a7d8361f290f3e888f"
server_key_iv: "4937b871e023609c6bb527d141663a1d"
server_key_version: "encryptionkey@0"
I20260812 21:12:55.354665 19116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:12:55.355501 19141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:55.355679 19116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:12:55.355777 19116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "5b480ca38e234b10bd44ebcb19a6763b"
format_stamp: "Formatted at 2026-08-12 21:12:55 on dist-test-slave-lkx2"
server_key: "a10ec7bfe8ba13a7d8361f290f3e888f"
server_key_iv: "4937b871e023609c6bb527d141663a1d"
server_key_version: "encryptionkey@0"
I20260812 21:12:55.355852 19116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:55.376991 19116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:55.379019 19116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:55.379145 19116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:55.383313 19116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38057
I20260812 21:12:55.383353 19234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38057 every 8 connection(s)
I20260812 21:12:55.383703 19116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:55.384600 19237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:55.386956 19237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b: Bootstrap starting.
I20260812 21:12:55.387552 19237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:55.387835 19237 log.cc:824] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:55.388623 19237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b: No bootstrap required, opened a new log
I20260812 21:12:55.390342 19237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b480ca38e234b10bd44ebcb19a6763b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38057 } }
I20260812 21:12:55.390486 19237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:55.390512 19237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b480ca38e234b10bd44ebcb19a6763b, State: Initialized, Role: FOLLOWER
I20260812 21:12:55.390666 19237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b480ca38e234b10bd44ebcb19a6763b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38057 } }
I20260812 21:12:55.390705 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19116
I20260812 21:12:55.390748 19237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:55.390789 19237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:55.390815 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:55.390837 19237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:55.391032 12094 external_mini_cluster.cc:1428] Setting key 8b24ed95c290398df21c35032514a2a5
I20260812 21:12:55.391489 19237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b480ca38e234b10bd44ebcb19a6763b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38057 } }
I20260812 21:12:55.391607 19237 leader_election.cc:302] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b480ca38e234b10bd44ebcb19a6763b; no voters:
I20260812 21:12:55.391812 19237 leader_election.cc:288] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:55.391880 19244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:55.392076 19244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [term 1 LEADER]: Becoming Leader. State: Replica: 5b480ca38e234b10bd44ebcb19a6763b, State: Running, Role: LEADER
I20260812 21:12:55.392105 19237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:55.392781 19244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b480ca38e234b10bd44ebcb19a6763b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38057 } }
I20260812 21:12:55.393414 19247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b480ca38e234b10bd44ebcb19a6763b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b480ca38e234b10bd44ebcb19a6763b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38057 } } }
I20260812 21:12:55.393533 19247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:55.393662 19249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b480ca38e234b10bd44ebcb19a6763b. Latest consensus state: current_term: 1 leader_uuid: "5b480ca38e234b10bd44ebcb19a6763b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b480ca38e234b10bd44ebcb19a6763b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38057 } } }
I20260812 21:12:55.393877 19249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b [sys.catalog]: This master's current role is: LEADER
W20260812 21:12:55.394441 19275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:12:55.394500 19275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:12:55.394557 19284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:55.394728 19284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:55.396508 19284 catalog_manager.cc:1347] Generated new cluster ID: 8ea0f147e0724e64882c8433cf7faa04
I20260812 21:12:55.396560 19284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:55.396939 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:55.391550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:58621 (local address 127.0.0.1:38057)
0812 21:12:55.391714 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:55.391724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:55.391762 (+ 38us) server_negotiation.cc:407] Connection header received
0812 21:12:55.391894 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:55.391902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:55.391967 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:55.392125 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:55.392370 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:55.394092 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:55.394988 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:55.395424 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:55.395524 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:55.396270 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:55.396274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:55.396275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:55.396500 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:55.396542 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:55.396558 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:55.396634 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:55.396729 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:12:55.396764 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260812 21:12:55.397821 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:55.408064 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:55.399086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:43045 (local address 127.0.0.1:38057)
0812 21:12:55.399121 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:55.399122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:55.404557 (+ 5435us) server_negotiation.cc:407] Connection header received
0812 21:12:55.404623 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:55.404624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:55.404634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:55.404693 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:55.404838 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:55.405580 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:55.406581 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:55.406887 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:55.406951 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:55.407705 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:55.407706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:55.407729 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:55.407739 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:55.407757 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:55.407761 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:55.407835 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:55.407922 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 21:12:55.407943 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:12:55.408157 19284 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:55.408797 19284 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:55.413780 19284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b480ca38e234b10bd44ebcb19a6763b: Generated new TSK 0
I20260812 21:12:55.413964 19284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:55.419746 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19116
W20260812 21:12:55.429824 19090 connection.cc:535] client connection to 127.0.0.1:38057 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38057): Connection reset by peer (error 104)
2026-08-12T21:12:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:55 dist-test-slave-lkx2 krb5kdc[19297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:55 dist-test-slave-lkx2 krb5kdc[19297](info): set up 2 sockets
Aug 12 21:12:55 dist-test-slave-lkx2 krb5kdc[19297](info): commencing operation
krb5kdc: starting...
W20260812 21:12:56.629943 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:56 dist-test-slave-lkx2 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:12:56.757295 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:56.840037 19318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:56.840284 19318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:56.840312 19318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:56.841531 19318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:56.841578 19318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:56.841590 19318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:56.841603 19318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:56.841614 19318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:56.843106 19318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46581
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:56.843330 19318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:56.843523 19318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:56.848374 19335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:56.848387 19327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:56.848402 19332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:56.848472 19318 server_base.cc:1034] running on GCE node
I20260812 21:12:56.848942 19318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:56.849251 19318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:56.850409 19318 hybrid_clock.cc:648] HybridClock initialized: now 1786569176850391 us; error 27 us; skew 500 ppm
Aug 12 21:12:56 dist-test-slave-lkx2 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:56.853021 19318 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:12:56.853515 19318 webserver.cc:460] Webserver started at http://127.0.0.1:32883/ using document root <none> and password file <none>
I20260812 21:12:56.853709 19318 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:56.853751 19318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:56.853849 19318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:56.854929 19318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "d5ebefdafc90497db71346557b8256f7"
format_stamp: "Formatted at 2026-08-12 21:12:56 on dist-test-slave-lkx2"
server_key: "85cd3f8511a5f52b76cf3435fae724f0"
server_key_iv: "3ffa9cefb91d2f2516c01800752e0de3"
server_key_version: "encryptionkey@0"
I20260812 21:12:56.855244 19318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "d5ebefdafc90497db71346557b8256f7"
format_stamp: "Formatted at 2026-08-12 21:12:56 on dist-test-slave-lkx2"
server_key: "85cd3f8511a5f52b76cf3435fae724f0"
server_key_iv: "3ffa9cefb91d2f2516c01800752e0de3"
server_key_version: "encryptionkey@0"
I20260812 21:12:56.856537 19318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:12:56.857227 19347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:56.857383 19318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:56.857481 19318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "d5ebefdafc90497db71346557b8256f7"
format_stamp: "Formatted at 2026-08-12 21:12:56 on dist-test-slave-lkx2"
server_key: "85cd3f8511a5f52b76cf3435fae724f0"
server_key_iv: "3ffa9cefb91d2f2516c01800752e0de3"
server_key_version: "encryptionkey@0"
I20260812 21:12:56.857544 19318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:56.860893 19318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:56.862926 19318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:56.863056 19318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:56.867004 19318 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46581
I20260812 21:12:56.867051 19443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46581 every 8 connection(s)
I20260812 21:12:56.867383 19318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:56.868448 19444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:56.870869 19444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7: Bootstrap starting.
I20260812 21:12:56.871459 19444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:56.871737 19444 log.cc:824] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:56.872503 19444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7: No bootstrap required, opened a new log
I20260812 21:12:56.873124 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19318
I20260812 21:12:56.873234 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:56.873433 12094 external_mini_cluster.cc:1428] Setting key afe715af3b8fdf015ce51e1fd0cd0eda
I20260812 21:12:56.874637 19444 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ebefdafc90497db71346557b8256f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260812 21:12:56.874845 19444 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:56.874886 19444 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ebefdafc90497db71346557b8256f7, State: Initialized, Role: FOLLOWER
I20260812 21:12:56.875073 19444 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ebefdafc90497db71346557b8256f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260812 21:12:56.875185 19444 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:56.875218 19444 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:56.875268 19444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:56.875946 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:56.874045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:54283 (local address 127.0.0.1:46581)
0812 21:12:56.874258 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:56.874269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:56.874323 (+ 54us) server_negotiation.cc:407] Connection header received
0812 21:12:56.874590 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:56.874600 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:56.874634 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:56.874788 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:56.875466 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:56.875471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:56.875471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:56.875476 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:56.875530 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:56.875544 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:56.875613 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:56.875665 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:12:56.875714 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260812 21:12:56.876484 19444 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ebefdafc90497db71346557b8256f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260812 21:12:56.876598 19444 leader_election.cc:302] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5ebefdafc90497db71346557b8256f7; no voters:
I20260812 21:12:56.876796 19444 leader_election.cc:288] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:56.876842 19454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:56.876984 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:56.877008 19454 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [term 1 LEADER]: Becoming Leader. State: Replica: d5ebefdafc90497db71346557b8256f7, State: Running, Role: LEADER
I20260812 21:12:56.877058 19444 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:56.877108 19454 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ebefdafc90497db71346557b8256f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260812 21:12:56.877486 19454 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ebefdafc90497db71346557b8256f7. Latest consensus state: current_term: 1 leader_uuid: "d5ebefdafc90497db71346557b8256f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ebefdafc90497db71346557b8256f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } } }
I20260812 21:12:56.877583 19454 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:56.877566 19456 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5ebefdafc90497db71346557b8256f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ebefdafc90497db71346557b8256f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } } }
I20260812 21:12:56.877677 19456 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:56.877911 19474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:56.878451 19474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:56.879906 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:56.878732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.231:44227 (local address 127.0.0.1:46581)
0812 21:12:56.878764 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:56.878766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:56.878963 (+ 197us) server_negotiation.cc:407] Connection header received
0812 21:12:56.879015 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:56.879016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:56.879022 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:56.879067 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:56.879680 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:56.879682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:56.879682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:56.879686 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:56.879702 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:56.879706 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:56.879769 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:56.879835 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 21:12:56.879849 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:12:56.880446 19474 catalog_manager.cc:1347] Generated new cluster ID: c1fc6548c62a4bbf8f026fb7e154331c
I20260812 21:12:56.880501 19474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:56.893323 19474 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:56.893537 19474 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:56.897862 19474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5ebefdafc90497db71346557b8256f7: Generated new TSK 0
I20260812 21:12:56.898064 19474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:56.900957 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19318
2026-08-12T21:12:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:56 dist-test-slave-lkx2 krb5kdc[19509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:56 dist-test-slave-lkx2 krb5kdc[19509](info): set up 2 sockets
Aug 12 21:12:56 dist-test-slave-lkx2 krb5kdc[19509](info): commencing operation
krb5kdc: starting...
W20260812 21:12:58.112478 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:58 dist-test-slave-lkx2 krb5kdc[19509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:12:58.239540 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44209
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:44209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:58.323225 19536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:58.323436 19536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:58.323458 19536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:58.324652 19536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:58.324695 19536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:58.324709 19536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:58.324720 19536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:58.324731 19536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:58.326227 19536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:44209
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:44209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:58.326478 19536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:58.326691 19536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:58.331590 19548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:58.331612 19550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:58.331595 19553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:58.331630 19536 server_base.cc:1034] running on GCE node
I20260812 21:12:58.332015 19536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:58.332312 19536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:58.333496 19536 hybrid_clock.cc:648] HybridClock initialized: now 1786569178333485 us; error 29 us; skew 500 ppm
Aug 12 21:12:58 dist-test-slave-lkx2 krb5kdc[19509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569178, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:58.341351 19536 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:12:58.341850 19536 webserver.cc:460] Webserver started at http://127.11.207.190:37927/ using document root <none> and password file <none>
I20260812 21:12:58.342083 19536 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:58.342137 19536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:58.342270 19536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:58.343364 19536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "a3938224d2804a6688801d9f644ea4d0"
format_stamp: "Formatted at 2026-08-12 21:12:58 on dist-test-slave-lkx2"
server_key: "0d853217122e57e231c519c79d2d0abe"
server_key_iv: "0cc7433d75a6966f601b3189d0c1f2ac"
server_key_version: "encryptionkey@0"
I20260812 21:12:58.343713 19536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "a3938224d2804a6688801d9f644ea4d0"
format_stamp: "Formatted at 2026-08-12 21:12:58 on dist-test-slave-lkx2"
server_key: "0d853217122e57e231c519c79d2d0abe"
server_key_iv: "0cc7433d75a6966f601b3189d0c1f2ac"
server_key_version: "encryptionkey@0"
I20260812 21:12:58.345139 19536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:12:58.345808 19562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:58.345968 19536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:12:58.346060 19536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "a3938224d2804a6688801d9f644ea4d0"
format_stamp: "Formatted at 2026-08-12 21:12:58 on dist-test-slave-lkx2"
server_key: "0d853217122e57e231c519c79d2d0abe"
server_key_iv: "0cc7433d75a6966f601b3189d0c1f2ac"
server_key_version: "encryptionkey@0"
I20260812 21:12:58.346120 19536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:58.359277 19536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:58.361156 19536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:58.361295 19536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:58.365347 19536 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:44209
I20260812 21:12:58.365401 19656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:44209 every 8 connection(s)
I20260812 21:12:58.365733 19536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:58.366618 19657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:58.369113 19657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0: Bootstrap starting.
I20260812 21:12:58.369712 19657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:58.369987 19657 log.cc:824] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:58.370658 19657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0: No bootstrap required, opened a new log
I20260812 21:12:58.372257 19657 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3938224d2804a6688801d9f644ea4d0" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44209 } }
I20260812 21:12:58.372466 19657 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:58.372507 19657 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3938224d2804a6688801d9f644ea4d0, State: Initialized, Role: FOLLOWER
I20260812 21:12:58.372653 19657 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3938224d2804a6688801d9f644ea4d0" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44209 } }
I20260812 21:12:58.372714 19657 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:58.372745 19657 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:58.372792 19657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:58.373646 19657 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3938224d2804a6688801d9f644ea4d0" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44209 } }
I20260812 21:12:58.373745 19657 leader_election.cc:302] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3938224d2804a6688801d9f644ea4d0; no voters:
I20260812 21:12:58.373899 19657 leader_election.cc:288] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:58.373936 19662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:58.374101 19662 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [term 1 LEADER]: Becoming Leader. State: Replica: a3938224d2804a6688801d9f644ea4d0, State: Running, Role: LEADER
I20260812 21:12:58.374164 19657 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:58.374209 19662 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3938224d2804a6688801d9f644ea4d0" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44209 } }
I20260812 21:12:58.374631 19666 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3938224d2804a6688801d9f644ea4d0. Latest consensus state: current_term: 1 leader_uuid: "a3938224d2804a6688801d9f644ea4d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3938224d2804a6688801d9f644ea4d0" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44209 } } }
I20260812 21:12:58.374733 19666 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:58.374617 19665 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3938224d2804a6688801d9f644ea4d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3938224d2804a6688801d9f644ea4d0" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 44209 } } }
I20260812 21:12:58.374933 19665 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0 [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:58.375018 19676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:58.375460 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19536
I20260812 21:12:58.375592 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:58.375597 19676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:58.375792 12094 external_mini_cluster.cc:1428] Setting key 27af183d38047dc81bef33edb7072094
I20260812 21:12:58.377439 19676 catalog_manager.cc:1347] Generated new cluster ID: e9a84960b802458c8d12aeb92eb1313c
I20260812 21:12:58.377496 19676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:12:58 dist-test-slave-lkx2 krb5kdc[19509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:12:58.385637 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:58.376391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36617 (local address 127.11.207.190:44209)
0812 21:12:58.376626 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:58.376635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:58.376667 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:12:58.376794 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:58.376801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:58.376856 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:58.377005 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:58.377669 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:58.378601 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:58.379347 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:58.379797 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:58.379901 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:58.382184 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:58.382187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:58.382188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:58.382200 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:58.384083 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:58.384490 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:58.384492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:58.384493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:58.384592 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:58.384807 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:58.384808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:58.384808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:58.385040 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:58.385208 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:58.385328 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 21:12:58.385442 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260812 21:12:58.386596 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:58.393822 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:58.387891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34749 (local address 127.11.207.190:44209)
0812 21:12:58.387926 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:58.387928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:58.388512 (+ 584us) server_negotiation.cc:407] Connection header received
0812 21:12:58.388567 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:58.388568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:58.388578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:58.388618 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:58.389107 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:58.389587 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:58.390245 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:58.390454 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:58.390509 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:58.391563 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:58.391564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:58.391564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:12:58.391571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:58.392778 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:58.393105 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:58.393106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:58.393106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:58.393157 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:12:58.393339 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:12:58.393340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:12:58.393340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:12:58.393454 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:58.393542 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:58.393676 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 21:12:58.393736 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:12:58.404950 19676 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:58.405556 19676 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:58.415843 19676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3938224d2804a6688801d9f644ea4d0: Generated new TSK 0
I20260812 21:12:58.416036 19676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:58.422286 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19536
W20260812 21:12:58.434109 19500 connection.cc:535] client connection to 127.11.207.190:44209 recv error: Network error: failed to read from TLS socket (remote: 127.11.207.190:44209): Connection reset by peer (error 104)
2026-08-12T21:12:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:58 dist-test-slave-lkx2 krb5kdc[19713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:58 dist-test-slave-lkx2 krb5kdc[19713](info): set up 2 sockets
Aug 12 21:12:58 dist-test-slave-lkx2 krb5kdc[19713](info): commencing operation
krb5kdc: starting...
W20260812 21:12:59.626631 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:12:59 dist-test-slave-lkx2 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:12:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:12:59.746954 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43591
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:43591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:12:59.833280 19742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:12:59.833509 19742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:12:59.833532 19742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:12:59.834702 19742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:12:59.834745 19742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:12:59.834758 19742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:12:59.834769 19742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:12:59.834780 19742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:12:59.836377 19742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:43591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:43591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:12:59.836627 19742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:12:59.836851 19742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:12:59.841629 19748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:59.841691 19749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:12:59.841629 19753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:12:59.842027 19742 server_base.cc:1034] running on GCE node
I20260812 21:12:59.842267 19742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:12:59.842581 19742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:12:59.843741 19742 hybrid_clock.cc:648] HybridClock initialized: now 1786569179843711 us; error 42 us; skew 500 ppm
Aug 12 21:12:59 dist-test-slave-lkx2 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569179, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:12:59.846289 19742 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:12:59.846764 19742 webserver.cc:460] Webserver started at http://127.11.207.190:44957/ using document root <none> and password file <none>
I20260812 21:12:59.846997 19742 fs_manager.cc:359] Metadata directory not provided
I20260812 21:12:59.847047 19742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:12:59.847183 19742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:12:59.848330 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "47bb22741d4441c9a2ff3a88f255fb8f"
format_stamp: "Formatted at 2026-08-12 21:12:59 on dist-test-slave-lkx2"
server_key: "872cf0925cb757199da6a266b6595ccc"
server_key_iv: "80f891d9ff7097d0551584c8d11de2fc"
server_key_version: "encryptionkey@0"
I20260812 21:12:59.848713 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "47bb22741d4441c9a2ff3a88f255fb8f"
format_stamp: "Formatted at 2026-08-12 21:12:59 on dist-test-slave-lkx2"
server_key: "872cf0925cb757199da6a266b6595ccc"
server_key_iv: "80f891d9ff7097d0551584c8d11de2fc"
server_key_version: "encryptionkey@0"
I20260812 21:12:59.849957 19742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:12:59.850638 19761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:12:59.850821 19742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:12:59.850905 19742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "47bb22741d4441c9a2ff3a88f255fb8f"
format_stamp: "Formatted at 2026-08-12 21:12:59 on dist-test-slave-lkx2"
server_key: "872cf0925cb757199da6a266b6595ccc"
server_key_iv: "80f891d9ff7097d0551584c8d11de2fc"
server_key_version: "encryptionkey@0"
I20260812 21:12:59.850972 19742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:12:59.873018 19742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:12:59.875283 19742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:12:59.875432 19742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:12:59.879516 19742 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:43591
I20260812 21:12:59.879554 19859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:43591 every 8 connection(s)
I20260812 21:12:59.879904 19742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:12:59.880838 19860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:12:59.882508 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19742
I20260812 21:12:59.882618 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:12:59.882810 12094 external_mini_cluster.cc:1428] Setting key ad06dab8769d7d33b78c884c9c7376e6
I20260812 21:12:59.883332 19860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f: Bootstrap starting.
I20260812 21:12:59.884006 19860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f: Neither blocks nor log segments found. Creating new log.
I20260812 21:12:59.884306 19860 log.cc:824] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:12:59.884989 19860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f: No bootstrap required, opened a new log
I20260812 21:12:59.887223 19860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43591 } }
I20260812 21:12:59.887435 19860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:12:59.887470 19860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bb22741d4441c9a2ff3a88f255fb8f, State: Initialized, Role: FOLLOWER
I20260812 21:12:59.887609 19860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43591 } }
I20260812 21:12:59.887696 19860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:12:59.887737 19860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:12:59.887778 19860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:12:59.888444 19860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43591 } }
I20260812 21:12:59.888566 19860 leader_election.cc:302] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bb22741d4441c9a2ff3a88f255fb8f; no voters:
I20260812 21:12:59.888639 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:59.883353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42349 (local address 127.11.207.190:43591)
0812 21:12:59.883528 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:59.883537 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:59.883563 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:12:59.884004 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:59.884011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:59.884065 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:59.884213 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:59.884483 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:59.885422 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:59.886541 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:59.886978 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:59.887085 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:59.887908 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:59.887912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:59.887913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:59.888085 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:59.888119 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:59.888129 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:59.888199 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:59.888324 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 21:12:59.888374 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260812 21:12:59.888746 19860 leader_election.cc:288] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:12:59.888882 19871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:12:59.889001 19860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:12:59.889070 19871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [term 1 LEADER]: Becoming Leader. State: Replica: 47bb22741d4441c9a2ff3a88f255fb8f, State: Running, Role: LEADER
I20260812 21:12:59.889165 19871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43591 } }
I20260812 21:12:59.889510 19871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bb22741d4441c9a2ff3a88f255fb8f. Latest consensus state: current_term: 1 leader_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43591 } } }
I20260812 21:12:59.889585 19871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:59.889743 19873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bb22741d4441c9a2ff3a88f255fb8f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 43591 } } }
I20260812 21:12:59.889802 19873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f [sys.catalog]: This master's current role is: LEADER
I20260812 21:12:59.889875 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:12:59.889912 19884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:12:59.890375 19884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:12:59.892033 19884 catalog_manager.cc:1347] Generated new cluster ID: 4971494b61c34ecb94418ad2c46b4f63
I20260812 21:12:59.892086 19884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:12:59.898805 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:12:59.891560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48535 (local address 127.11.207.190:43591)
0812 21:12:59.891604 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:12:59.891606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:12:59.891749 (+ 143us) server_negotiation.cc:407] Connection header received
0812 21:12:59.891795 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:12:59.891796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:12:59.891806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:12:59.891868 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:12:59.895726 (+ 3858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:59.896499 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:59.897524 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:12:59.897836 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:12:59.897893 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:12:59.898575 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:12:59.898576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:12:59.898576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:12:59.898580 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:12:59.898597 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:12:59.898602 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:12:59.898653 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:12:59.898724 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:12:59.898733 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:12:59.908099 19884 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:12:59.908650 19884 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:12:59.911764 19884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47bb22741d4441c9a2ff3a88f255fb8f: Generated new TSK 0
I20260812 21:12:59.911917 19884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:12:59.915264 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19742
W20260812 21:12:59.934988 19708 connection.cc:535] client connection to 127.11.207.190:43591 recv error: Network error: failed to read from TLS socket (remote: 127.11.207.190:43591): Connection reset by peer (error 104)
2026-08-12T21:12:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:12:59 dist-test-slave-lkx2 krb5kdc[19917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:12:59 dist-test-slave-lkx2 krb5kdc[19917](info): set up 2 sockets
Aug 12 21:12:59 dist-test-slave-lkx2 krb5kdc[19917](info): commencing operation
krb5kdc: starting...
W20260812 21:13:01.142053 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:13:01 dist-test-slave-lkx2 krb5kdc[19917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:13:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:13:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:13:01.263304 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34949
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:13:01.363688 19940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:01.363903 19940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:01.363934 19940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:01.365131 19940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:01.365177 19940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:01.365197 19940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:01.365214 19940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:01.365231 19940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:01.366727 19940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:01.366981 19940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:01.367190 19940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:01.371809 19950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:01.371894 19954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:01.371829 19951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:01.372080 19940 server_base.cc:1034] running on GCE node
I20260812 21:13:01.372314 19940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:01.372622 19940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:01.373759 19940 hybrid_clock.cc:648] HybridClock initialized: now 1786569181373741 us; error 37 us; skew 500 ppm
Aug 12 21:13:01 dist-test-slave-lkx2 krb5kdc[19917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569181, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:01.376332 19940 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:13:01.376811 19940 webserver.cc:460] Webserver started at http://127.11.207.190:43697/ using document root <none> and password file <none>
I20260812 21:13:01.377032 19940 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:01.377082 19940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:01.377219 19940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:01.378254 19940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "b00a2d20d1e84d0cbc8546239e35852a"
format_stamp: "Formatted at 2026-08-12 21:13:01 on dist-test-slave-lkx2"
server_key: "c82d33ef232a91886d578e7dce4705f9"
server_key_iv: "79b7d3e206b978b516fb29fb3b3ce220"
server_key_version: "encryptionkey@0"
I20260812 21:13:01.378619 19940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "b00a2d20d1e84d0cbc8546239e35852a"
format_stamp: "Formatted at 2026-08-12 21:13:01 on dist-test-slave-lkx2"
server_key: "c82d33ef232a91886d578e7dce4705f9"
server_key_iv: "79b7d3e206b978b516fb29fb3b3ce220"
server_key_version: "encryptionkey@0"
I20260812 21:13:01.379882 19940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:13:01.380573 19970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:01.380725 19940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:13:01.380820 19940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "b00a2d20d1e84d0cbc8546239e35852a"
format_stamp: "Formatted at 2026-08-12 21:13:01 on dist-test-slave-lkx2"
server_key: "c82d33ef232a91886d578e7dce4705f9"
server_key_iv: "79b7d3e206b978b516fb29fb3b3ce220"
server_key_version: "encryptionkey@0"
I20260812 21:13:01.380877 19940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:01.384147 19940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:01.385887 19940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:01.386013 19940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:01.389882 19940 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:34949
I20260812 21:13:01.389916 20060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:34949 every 8 connection(s)
I20260812 21:13:01.390210 19940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:13:01.391171 20061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:01.393460 20061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a: Bootstrap starting.
I20260812 21:13:01.393990 20061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:01.394219 20061 log.cc:824] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:01.394821 20061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a: No bootstrap required, opened a new log
I20260812 21:13:01.396415 20061 raft_consensus.cc:348] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00a2d20d1e84d0cbc8546239e35852a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34949 } }
I20260812 21:13:01.396559 20061 raft_consensus.cc:374] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:13:01.396581 20061 raft_consensus.cc:729] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b00a2d20d1e84d0cbc8546239e35852a, State: Initialized, Role: FOLLOWER
I20260812 21:13:01.396689 20061 consensus_queue.cc:260] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00a2d20d1e84d0cbc8546239e35852a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34949 } }
I20260812 21:13:01.396759 20061 raft_consensus.cc:388] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:13:01.396790 20061 raft_consensus.cc:482] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:13:01.396822 20061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:01.397607 20061 raft_consensus.cc:504] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00a2d20d1e84d0cbc8546239e35852a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34949 } }
I20260812 21:13:01.397698 20061 leader_election.cc:302] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b00a2d20d1e84d0cbc8546239e35852a; no voters:
I20260812 21:13:01.397822 20061 leader_election.cc:288] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:13:01.397876 20067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:13:01.398033 20067 raft_consensus.cc:686] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [term 1 LEADER]: Becoming Leader. State: Replica: b00a2d20d1e84d0cbc8546239e35852a, State: Running, Role: LEADER
I20260812 21:13:01.398074 20061 sys_catalog.cc:564] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:01.398125 20067 consensus_queue.cc:237] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00a2d20d1e84d0cbc8546239e35852a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34949 } }
I20260812 21:13:01.398506 20069 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b00a2d20d1e84d0cbc8546239e35852a. Latest consensus state: current_term: 1 leader_uuid: "b00a2d20d1e84d0cbc8546239e35852a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00a2d20d1e84d0cbc8546239e35852a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34949 } } }
I20260812 21:13:01.398501 20068 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b00a2d20d1e84d0cbc8546239e35852a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00a2d20d1e84d0cbc8546239e35852a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34949 } } }
I20260812 21:13:01.398598 20069 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:01.398648 20068 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:01.398865 20085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:13:01.399012 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 19940
I20260812 21:13:01.399122 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:13:01.399302 12094 external_mini_cluster.cc:1428] Setting key e20719c50900bba2477da457e46d2fd3
I20260812 21:13:01.399442 20085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:13:01.401636 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:01.400028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50447 (local address 127.11.207.190:34949)
0812 21:13:01.400187 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:01.400196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:01.400223 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:13:01.400426 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:01.400434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:01.400466 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:01.400602 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:01.401257 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:01.401261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:01.401261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:13:01.401265 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:01.401305 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:13:01.401323 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:01.401376 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:01.401447 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:13:01.401490 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260812 21:13:01.401722 20085 catalog_manager.cc:1347] Generated new cluster ID: 61e64e6740a440de96f76ca93815af02
I20260812 21:13:01.401862 20085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:13:01.402695 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:13:01.405311 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:01.404136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33569 (local address 127.11.207.190:34949)
0812 21:13:01.404188 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:01.404190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:01.404419 (+ 229us) server_negotiation.cc:407] Connection header received
0812 21:13:01.404480 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:01.404481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:01.404486 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:01.404535 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:01.405131 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:01.405131 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:01.405132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:13:01.405134 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:01.405146 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:13:01.405149 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:01.405189 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:01.405243 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 21:13:01.405254 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:13:01.416409 20085 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:13:01.416563 20085 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:13:01.419644 20085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b00a2d20d1e84d0cbc8546239e35852a: Generated new TSK 0
I20260812 21:13:01.419804 20085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:13:01.421233 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 19940
2026-08-12T21:13:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:13:01 dist-test-slave-lkx2 krb5kdc[20120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:13:01 dist-test-slave-lkx2 krb5kdc[20120](info): set up 2 sockets
Aug 12 21:13:01 dist-test-slave-lkx2 krb5kdc[20120](info): commencing operation
krb5kdc: starting...
W20260812 21:13:02.615936 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:13:02 dist-test-slave-lkx2 krb5kdc[20120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:13:02.735461 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:45657
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:37933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:45657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:13:02.818964 20143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:02.819177 20143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:02.819208 20143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:02.820394 20143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:02.820437 20143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:02.820456 20143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:02.820472 20143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:02.820485 20143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:02.822005 20143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:37933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:45657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:45657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:02.822261 20143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:02.822480 20143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:02.827191 20150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:02.827165 20161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:02.827280 20143 server_base.cc:1034] running on GCE node
W20260812 21:13:02.827406 20151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:02.827674 20143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:02.827946 20143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:02.829095 20143 hybrid_clock.cc:648] HybridClock initialized: now 1786569182829079 us; error 32 us; skew 500 ppm
Aug 12 21:13:02 dist-test-slave-lkx2 krb5kdc[20120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569182, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:02.831552 20143 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:13:02.832000 20143 webserver.cc:460] Webserver started at http://127.11.207.190:34619/ using document root <none> and password file <none>
I20260812 21:13:02.832204 20143 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:02.832253 20143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:02.832425 20143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:02.833508 20143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "0696a72697294a83a6464db9193b5057"
format_stamp: "Formatted at 2026-08-12 21:13:02 on dist-test-slave-lkx2"
server_key: "b620d0a12428a559c1adc9351d2658fb"
server_key_iv: "954e79ca60eabdbd1aed2f191739b1e4"
server_key_version: "encryptionkey@0"
I20260812 21:13:02.833853 20143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "0696a72697294a83a6464db9193b5057"
format_stamp: "Formatted at 2026-08-12 21:13:02 on dist-test-slave-lkx2"
server_key: "b620d0a12428a559c1adc9351d2658fb"
server_key_iv: "954e79ca60eabdbd1aed2f191739b1e4"
server_key_version: "encryptionkey@0"
I20260812 21:13:02.835150 20143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:13:02.835846 20171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:02.836009 20143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:13:02.836099 20143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "0696a72697294a83a6464db9193b5057"
format_stamp: "Formatted at 2026-08-12 21:13:02 on dist-test-slave-lkx2"
server_key: "b620d0a12428a559c1adc9351d2658fb"
server_key_iv: "954e79ca60eabdbd1aed2f191739b1e4"
server_key_version: "encryptionkey@0"
I20260812 21:13:02.836169 20143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:02.849370 20143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:02.851157 20143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:02.851302 20143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:02.855348 20143 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:45657
I20260812 21:13:02.855383 20252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:45657 every 8 connection(s)
I20260812 21:13:02.855731 20143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:13:02.856638 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:02.858976 20254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057: Bootstrap starting.
I20260812 21:13:02.859547 20254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:02.859778 20254 log.cc:824] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:02.860469 20254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057: No bootstrap required, opened a new log
I20260812 21:13:02.860910 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20143
I20260812 21:13:02.861022 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:13:02.861218 12094 external_mini_cluster.cc:1428] Setting key 9c0afa8b0e028f73eb87e31f370c72d1
I20260812 21:13:02.862998 20254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0696a72697294a83a6464db9193b5057" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45657 } }
I20260812 21:13:02.863169 20254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:13:02.863206 20254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0696a72697294a83a6464db9193b5057, State: Initialized, Role: FOLLOWER
I20260812 21:13:02.863338 20254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0696a72697294a83a6464db9193b5057" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45657 } }
I20260812 21:13:02.863452 20254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:13:02.863515 20254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:13:02.863567 20254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:02.864207 20254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0696a72697294a83a6464db9193b5057" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45657 } }
I20260812 21:13:02.864300 20254 leader_election.cc:302] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0696a72697294a83a6464db9193b5057; no voters:
I20260812 21:13:02.864488 20254 leader_election.cc:288] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:13:02.864533 20262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:13:02.864709 20262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [term 1 LEADER]: Becoming Leader. State: Replica: 0696a72697294a83a6464db9193b5057, State: Running, Role: LEADER
I20260812 21:13:02.864768 20254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:02.864799 20262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0696a72697294a83a6464db9193b5057" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45657 } }
I20260812 21:13:02.865197 20266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0696a72697294a83a6464db9193b5057. Latest consensus state: current_term: 1 leader_uuid: "0696a72697294a83a6464db9193b5057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0696a72697294a83a6464db9193b5057" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45657 } } }
I20260812 21:13:02.865338 20266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:02.865769 20263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0696a72697294a83a6464db9193b5057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0696a72697294a83a6464db9193b5057" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 45657 } } }
I20260812 21:13:02.865901 20263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057 [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:02.866047 20277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:13:02.866571 20277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:13:02.868185 20277 catalog_manager.cc:1347] Generated new cluster ID: 454182a84a054b039e11cf9ff8cac0c0
I20260812 21:13:02.868242 20277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:13:02 dist-test-slave-lkx2 krb5kdc[20120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:13:02.872216 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:02.861729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55489 (local address 127.11.207.190:45657)
0812 21:13:02.861908 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:02.861915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:02.861941 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:13:02.862923 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:02.862931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:02.862985 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:02.863100 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:02.863906 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:02.865132 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:02.866163 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:02.866755 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:02.866872 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:02.869281 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:02.869286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:02.869286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:02.869299 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:02.870811 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:02.871132 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:02.871136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:02.871136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:02.871219 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:02.871408 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:02.871409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:02.871409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:02.871634 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:02.871793 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:02.871887 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 21:13:02.872021 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260812 21:13:02.873095 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:13:02.875442 20277 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:13:02.876157 20277 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:13:02.880896 20277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0696a72697294a83a6464db9193b5057: Generated new TSK 0
I20260812 21:13:02.880965 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:02.875234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48043 (local address 127.11.207.190:45657)
0812 21:13:02.875276 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:02.875278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:02.875419 (+ 141us) server_negotiation.cc:407] Connection header received
0812 21:13:02.875521 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:02.875521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:02.875531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:02.875558 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:02.876079 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:02.876625 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:02.877280 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:02.877512 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:02.877566 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:02.878601 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:02.878602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:02.878602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:02.878609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:02.879818 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:02.880124 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:02.880125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:02.880125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:02.880176 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:02.880336 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:02.880336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:02.880337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:02.880468 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:02.880553 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:02.880707 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 21:13:02.880812 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:13:02.881120 20277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:13:02.885464 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20143
2026-08-12T21:13:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:13:02 dist-test-slave-lkx2 krb5kdc[20311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:13:02 dist-test-slave-lkx2 krb5kdc[20311](info): set up 2 sockets
Aug 12 21:13:02 dist-test-slave-lkx2 krb5kdc[20311](info): commencing operation
krb5kdc: starting...
W20260812 21:13:04.100886 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:13:04 dist-test-slave-lkx2 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:13:04.224852 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41495
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:46817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:13:04.307471 20329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:04.307694 20329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:04.307736 20329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:04.308950 20329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:04.308993 20329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:04.309012 20329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:04.309027 20329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:04.309042 20329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:04.310624 20329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:46817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:04.310883 20329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:04.311100 20329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:04.316118 20337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:04.316118 20329 server_base.cc:1034] running on GCE node
W20260812 21:13:04.316102 20338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:04.316071 20341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:04.316568 20329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:04.316846 20329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:04.317979 20329 hybrid_clock.cc:648] HybridClock initialized: now 1786569184317964 us; error 34 us; skew 500 ppm
Aug 12 21:13:04 dist-test-slave-lkx2 krb5kdc[20311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569184, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:04.325702 20329 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:13:04.326188 20329 webserver.cc:460] Webserver started at http://127.11.207.190:43053/ using document root <none> and password file <none>
I20260812 21:13:04.326439 20329 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:04.326491 20329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:04.326618 20329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:04.327708 20329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "dc642bc5581444329d3136605705b12c"
format_stamp: "Formatted at 2026-08-12 21:13:04 on dist-test-slave-lkx2"
server_key: "df1c3f19ffb25b11c403210954e75aa4"
server_key_iv: "a1516317fcaa2f59eb585599040b4975"
server_key_version: "encryptionkey@0"
I20260812 21:13:04.328042 20329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "dc642bc5581444329d3136605705b12c"
format_stamp: "Formatted at 2026-08-12 21:13:04 on dist-test-slave-lkx2"
server_key: "df1c3f19ffb25b11c403210954e75aa4"
server_key_iv: "a1516317fcaa2f59eb585599040b4975"
server_key_version: "encryptionkey@0"
I20260812 21:13:04.329402 20329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:13:04.330129 20352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:04.330281 20329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:04.330384 20329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "dc642bc5581444329d3136605705b12c"
format_stamp: "Formatted at 2026-08-12 21:13:04 on dist-test-slave-lkx2"
server_key: "df1c3f19ffb25b11c403210954e75aa4"
server_key_iv: "a1516317fcaa2f59eb585599040b4975"
server_key_version: "encryptionkey@0"
I20260812 21:13:04.330447 20329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:04.340394 20329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:04.342155 20329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:04.342300 20329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:04.346271 20329 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41495
I20260812 21:13:04.346321 20450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41495 every 8 connection(s)
I20260812 21:13:04.346663 20329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:13:04.347556 20451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:04.349951 20451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c: Bootstrap starting.
I20260812 21:13:04.350167 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20329
I20260812 21:13:04.350267 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:13:04.350440 12094 external_mini_cluster.cc:1428] Setting key f5361533d598713bee290b237ecd708e
I20260812 21:13:04.350627 20451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:04.350970 20451 log.cc:824] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:04.351771 20451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c: No bootstrap required, opened a new log
I20260812 21:13:04.354168 20451 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc642bc5581444329d3136605705b12c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41495 } }
I20260812 21:13:04.354312 20451 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:13:04.354346 20451 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc642bc5581444329d3136605705b12c, State: Initialized, Role: FOLLOWER
I20260812 21:13:04.354508 20451 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc642bc5581444329d3136605705b12c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41495 } }
I20260812 21:13:04.354589 20451 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:13:04.354626 20451 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:13:04.354669 20451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:04.355378 20451 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc642bc5581444329d3136605705b12c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41495 } }
I20260812 21:13:04.355491 20451 leader_election.cc:302] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc642bc5581444329d3136605705b12c; no voters:
I20260812 21:13:04.355675 20451 leader_election.cc:288] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:13:04.355700 20456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:13:04.355872 20456 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [term 1 LEADER]: Becoming Leader. State: Replica: dc642bc5581444329d3136605705b12c, State: Running, Role: LEADER
I20260812 21:13:04.356010 20451 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:04.356024 20456 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc642bc5581444329d3136605705b12c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41495 } }
I20260812 21:13:04.356503 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:04.350967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33867 (local address 127.11.207.190:41495)
0812 21:13:04.351174 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:04.351184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:04.351219 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:13:04.351754 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:04.351764 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:04.351831 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:04.352005 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:04.352206 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:04.353651 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:04.354412 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:04.355017 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:04.355139 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:04.355771 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:04.355775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:04.355776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:13:04.355998 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:04.356040 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:13:04.356056 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:04.356116 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:04.356197 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 21:13:04.356262 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260812 21:13:04.356755 20457 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc642bc5581444329d3136605705b12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc642bc5581444329d3136605705b12c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41495 } } }
I20260812 21:13:04.356779 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc642bc5581444329d3136605705b12c. Latest consensus state: current_term: 1 leader_uuid: "dc642bc5581444329d3136605705b12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc642bc5581444329d3136605705b12c" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41495 } } }
I20260812 21:13:04.356918 20457 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:04.356931 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:04.357278 20472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:13:04.357493 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:13:04.359125 20472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:13:04.360479 20472 catalog_manager.cc:1347] Generated new cluster ID: 3a68ee979dd64ea08cd50082a4e01ac9
I20260812 21:13:04.360526 20472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:13:04.362416 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:04.358783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36337 (local address 127.11.207.190:41495)
0812 21:13:04.358828 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:04.358829 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:04.359036 (+ 207us) server_negotiation.cc:407] Connection header received
0812 21:13:04.359128 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:04.359129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:04.359138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:04.359165 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:04.359349 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:04.360046 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:04.361041 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:04.361327 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:04.361369 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:04.362167 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:04.362169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:04.362169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:13:04.362174 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:04.362188 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:13:04.362193 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:04.362218 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:04.362318 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 21:13:04.362342 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:13:04.372694 20472 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:13:04.373464 20472 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:13:04.377411 20472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc642bc5581444329d3136605705b12c: Generated new TSK 0
I20260812 21:13:04.377595 20472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:13:04.384850 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20329
2026-08-12T21:13:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:13:04 dist-test-slave-lkx2 krb5kdc[20506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:13:04 dist-test-slave-lkx2 krb5kdc[20506](info): set up 2 sockets
Aug 12 21:13:04 dist-test-slave-lkx2 krb5kdc[20506](info): commencing operation
krb5kdc: starting...
W20260812 21:13:05.603711 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:13:05 dist-test-slave-lkx2 krb5kdc[20506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:13:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:13:05.726605 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41477
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:41477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:13:05.811234 20529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:05.811458 20529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:05.811480 20529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:05.812676 20529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:05.812718 20529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:05.812731 20529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:05.812742 20529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:05.812757 20529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:05.814273 20529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:41477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:41477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:05.814512 20529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:05.814718 20529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:05.819643 20537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:05.819736 20529 server_base.cc:1034] running on GCE node
W20260812 21:13:05.819730 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:05.819674 20542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:05.820088 20529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:05.820382 20529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:05.821530 20529 hybrid_clock.cc:648] HybridClock initialized: now 1786569185821512 us; error 35 us; skew 500 ppm
Aug 12 21:13:05 dist-test-slave-lkx2 krb5kdc[20506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569185, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:05.823906 20529 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:13:05.824370 20529 webserver.cc:460] Webserver started at http://127.11.207.190:46127/ using document root <none> and password file <none>
I20260812 21:13:05.824580 20529 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:05.824625 20529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:05.824765 20529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:05.825853 20529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "de8a4942ba9d430ca37fb5eff74b84fc"
format_stamp: "Formatted at 2026-08-12 21:13:05 on dist-test-slave-lkx2"
server_key: "ffd115138e814fedecc00cdb8de5adc1"
server_key_iv: "06ce1e0b0124163a91fbc6d5cd661a37"
server_key_version: "encryptionkey@0"
I20260812 21:13:05.826200 20529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "de8a4942ba9d430ca37fb5eff74b84fc"
format_stamp: "Formatted at 2026-08-12 21:13:05 on dist-test-slave-lkx2"
server_key: "ffd115138e814fedecc00cdb8de5adc1"
server_key_iv: "06ce1e0b0124163a91fbc6d5cd661a37"
server_key_version: "encryptionkey@0"
I20260812 21:13:05.827504 20529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:13:05.828213 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:05.828411 20529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:13:05.828502 20529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "de8a4942ba9d430ca37fb5eff74b84fc"
format_stamp: "Formatted at 2026-08-12 21:13:05 on dist-test-slave-lkx2"
server_key: "ffd115138e814fedecc00cdb8de5adc1"
server_key_iv: "06ce1e0b0124163a91fbc6d5cd661a37"
server_key_version: "encryptionkey@0"
I20260812 21:13:05.828559 20529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:05.831823 20529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:05.833948 20529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:05.834059 20529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:05.837966 20529 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:41477
I20260812 21:13:05.837999 20650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:41477 every 8 connection(s)
I20260812 21:13:05.838316 20529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:13:05.839293 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:05.841662 20651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc: Bootstrap starting.
I20260812 21:13:05.841768 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20529
I20260812 21:13:05.841897 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:13:05.842077 12094 external_mini_cluster.cc:1428] Setting key d5fb3f39a4ab65c7c6ea26f1a7cf87eb
I20260812 21:13:05.842295 20651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:05.842618 20651 log.cc:824] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:05.843379 20651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc: No bootstrap required, opened a new log
I20260812 21:13:05.844735 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:05.842552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52357 (local address 127.11.207.190:41477)
0812 21:13:05.842763 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:05.842773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:05.842900 (+ 127us) server_negotiation.cc:407] Connection header received
0812 21:13:05.843485 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:05.843494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:05.843522 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:05.843639 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:05.844265 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:05.844268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:05.844269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:13:05.844272 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:05.844309 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:13:05.844320 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:05.844441 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:05.844511 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 21:13:05.844546 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260812 21:13:05.845703 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:13:05.845783 20651 raft_consensus.cc:348] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41477 } }
I20260812 21:13:05.845947 20651 raft_consensus.cc:374] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:13:05.845983 20651 raft_consensus.cc:729] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8a4942ba9d430ca37fb5eff74b84fc, State: Initialized, Role: FOLLOWER
I20260812 21:13:05.846099 20651 consensus_queue.cc:260] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41477 } }
I20260812 21:13:05.846187 20651 raft_consensus.cc:388] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:13:05.846226 20651 raft_consensus.cc:482] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:13:05.846272 20651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:05.846992 20651 raft_consensus.cc:504] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41477 } }
I20260812 21:13:05.847150 20651 leader_election.cc:302] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de8a4942ba9d430ca37fb5eff74b84fc; no voters:
I20260812 21:13:05.847380 20651 leader_election.cc:288] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:13:05.847401 20671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:13:05.847533 20671 raft_consensus.cc:686] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [term 1 LEADER]: Becoming Leader. State: Replica: de8a4942ba9d430ca37fb5eff74b84fc, State: Running, Role: LEADER
I20260812 21:13:05.847632 20671 consensus_queue.cc:237] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41477 } }
I20260812 21:13:05.847687 20651 sys_catalog.cc:564] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:05.848219 20671 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader de8a4942ba9d430ca37fb5eff74b84fc. Latest consensus state: current_term: 1 leader_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41477 } } }
I20260812 21:13:05.848476 20671 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:05.848567 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:05.847281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48171 (local address 127.11.207.190:41477)
0812 21:13:05.847321 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:05.847322 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:05.847452 (+ 130us) server_negotiation.cc:407] Connection header received
0812 21:13:05.847539 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:05.847540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:05.847545 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:05.847602 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:05.848300 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:05.848301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:05.848302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:13:05.848306 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:05.848322 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:13:05.848327 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:05.848379 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:05.848494 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 21:13:05.848510 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:13:05.848397 20672 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8a4942ba9d430ca37fb5eff74b84fc" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 41477 } } }
I20260812 21:13:05.848783 20672 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc [sys.catalog]: This master's current role is: LEADER
W20260812 21:13:05.849537 20696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:05.849586 20696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:05.849656 20695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:13:05.849792 20695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:13:05.851104 20695 catalog_manager.cc:1347] Generated new cluster ID: 8870246581fc4eb4a1e6f92e152f5c58
I20260812 21:13:05.851156 20695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:13:05.868517 20695 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:13:05.868711 20695 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:13:05.881781 20695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de8a4942ba9d430ca37fb5eff74b84fc: Generated new TSK 0
I20260812 21:13:05.882001 20695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:13:05.891062 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20529
2026-08-12T21:13:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1495 ms)
[----------] 18 tests from AuthTokenIssuingTest (26962 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:13:05 dist-test-slave-lkx2 krb5kdc[20703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:13:05 dist-test-slave-lkx2 krb5kdc[20703](info): set up 2 sockets
Aug 12 21:13:05 dist-test-slave-lkx2 krb5kdc[20703](info): commencing operation
krb5kdc: starting...
W20260812 21:13:07.094246 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:13:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:13:07.216429 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:35127
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:35127,127.11.207.189:33073,127.11.207.188:40499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:13:07.298525 20736 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:13:07.298751 20736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:07.298776 20736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:07.298803 20736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:07.299999 20736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:07.300045 20736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:07.300063 20736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:07.300078 20736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:07.300094 20736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:07.301677 20736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:35127,127.11.207.189:33073,127.11.207.188:40499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:07.301950 20736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:07.302170 20736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:07.307035 20747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:07.307035 20751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:07.307137 20749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:07.307301 20736 server_base.cc:1034] running on GCE node
I20260812 21:13:07.307529 20736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:07.307814 20736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:07.308954 20736 hybrid_clock.cc:648] HybridClock initialized: now 1786569187308943 us; error 30 us; skew 500 ppm
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:07.311357 20736 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:13:07.311748 20736 webserver.cc:460] Webserver started at http://127.11.207.190:45221/ using document root <none> and password file <none>
I20260812 21:13:07.311959 20736 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:07.311998 20736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:07.312090 20736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:07.313189 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "cd7f17a47b294b29842e29fe0526153f"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "bd439b513a74a4bdf32b14745bfad7b3"
server_key_iv: "8455917a734836359133bbfe9056ed6f"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.313503 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "cd7f17a47b294b29842e29fe0526153f"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "bd439b513a74a4bdf32b14745bfad7b3"
server_key_iv: "8455917a734836359133bbfe9056ed6f"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.314700 20736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:13:07.315358 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:07.315515 20736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:13:07.315594 20736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "cd7f17a47b294b29842e29fe0526153f"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "bd439b513a74a4bdf32b14745bfad7b3"
server_key_iv: "8455917a734836359133bbfe9056ed6f"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.315681 20736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:07.334941 20736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:07.336817 20736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:07.336928 20736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:07.340771 20736 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:35127
I20260812 21:13:07.340831 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:35127 every 8 connection(s)
I20260812 21:13:07.341153 20736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:13:07.341691 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20736
I20260812 21:13:07.341807 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:13:07.342022 12094 external_mini_cluster.cc:1428] Setting key 9769b17b105e8e97d9013e5e71d0fd99
I20260812 21:13:07.342232 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:07.344274 20864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.207.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.189@KRBTEST.COM" created.
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:13:07.356299 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.345052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:35127 (local address 127.0.0.1:36748)
0812 21:13:07.345237 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.345276 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.345757 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.346016 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.346019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.346296 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.346486 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.346492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.347472 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.347473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.348142 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.348144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.348663 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.348665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.348732 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.349227 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.349236 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.353021 (+ 3785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.354762 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.354770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.354772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.355047 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.355258 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.355258 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.355259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.355369 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.355824 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.355829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.355890 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.355960 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.356094 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260812 21:13:07.356329 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.345060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36748 (local address 127.11.207.190:35127)
0812 21:13:07.345238 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.345248 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.345365 (+ 117us) server_negotiation.cc:407] Connection header received
0812 21:13:07.345911 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.345913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.345932 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.346007 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.346576 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.347387 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.348217 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.348583 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.348675 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.353120 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.353130 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.353130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.353139 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.354678 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.355121 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.355123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.355124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.355202 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.355442 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.355443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.355443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.355738 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.355822 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.356019 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.356180 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260812 21:13:07.359396 20864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:07.359730 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.359626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50120)
0812 21:13:07.359667 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.359708 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:13:07.359849 20760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
I20260812 21:13:07.360251 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.360176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50126)
0812 21:13:07.360210 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.360234 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:13:07.360416 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:33073: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab.
Entry for principal kudu/127.11.207.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab.
I20260812 21:13:07.366588 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:33073
--webserver_interface=127.11.207.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab
--principal=kudu/127.11.207.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:35127,127.11.207.189:33073,127.11.207.188:40499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:13:07.409631 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } attempt: 1
I20260812 21:13:07.410199 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.410060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50132)
0812 21:13:07.410119 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.410167 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:13:07.410542 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.410459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50134)
0812 21:13:07.410497 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.410522 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:13:07.410643 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:33073: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
W20260812 21:13:07.449729 20875 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:13:07.449960 20875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:07.449977 20875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:07.449995 20875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:07.451105 20875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:07.451138 20875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:07.451151 20875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:07.451162 20875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:07.451172 20875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:07.452703 20875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:35127,127.11.207.189:33073,127.11.207.188:40499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab
--principal=kudu/127.11.207.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.207.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:07.452924 20875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:07.453104 20875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:07.457794 20886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:07.457825 20883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:07.457839 20884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:07.458055 20875 server_base.cc:1034] running on GCE node
I20260812 21:13:07.458274 20875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:07.458535 20875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:07.459671 20875 hybrid_clock.cc:648] HybridClock initialized: now 1786569187459659 us; error 29 us; skew 500 ppm
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:07.462111 20875 init.cc:377] Logged in from keytab as kudu/127.11.207.189@KRBTEST.COM (short username kudu)
I20260812 21:13:07.462522 20875 webserver.cc:460] Webserver started at http://127.11.207.189:36843/ using document root <none> and password file <none>
I20260812 21:13:07.462724 20875 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:07.462767 20875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:07.462901 20875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:07.463948 20875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data/instance:
uuid: "d51623df83d442568d7dbcf059e648da"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "2ad60fc9c37fadc913756032a47fd775"
server_key_iv: "73476b20170c399c2b095f2823fb1e44"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.464272 20875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal/instance:
uuid: "d51623df83d442568d7dbcf059e648da"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "2ad60fc9c37fadc913756032a47fd775"
server_key_iv: "73476b20170c399c2b095f2823fb1e44"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.465528 20875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:13:07.466185 20896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:07.466352 20875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:13:07.466442 20875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal
uuid: "d51623df83d442568d7dbcf059e648da"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "2ad60fc9c37fadc913756032a47fd775"
server_key_iv: "73476b20170c399c2b095f2823fb1e44"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.466500 20875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:07.474783 20875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:07.476644 20875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:07.476752 20875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:07.478806 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } attempt: 2
I20260812 21:13:07.479259 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.479101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50138)
0812 21:13:07.479170 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.479218 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:13:07.479614 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.479507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50140)
0812 21:13:07.479557 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.479590 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 21:13:07.479705 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:33073: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33073: connect: Connection refused (error 111)
I20260812 21:13:07.480885 20875 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.189:33073
I20260812 21:13:07.480927 20984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.189:33073 every 8 connection(s)
I20260812 21:13:07.481253 20875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
I20260812 21:13:07.482152 20986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:07.482191 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20875
I20260812 21:13:07.482275 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-1/wal/instance
I20260812 21:13:07.482489 12094 external_mini_cluster.cc:1428] Setting key 00fc25e3e95587e3395f4a188e55fd5f
I20260812 21:13:07.484191 20986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.188@KRBTEST.COM; defaulting to no policy
I20260812 21:13:07.492667 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.485004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.11.207.190:35127)
0812 21:13:07.485049 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.485051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.485283 (+ 232us) server_negotiation.cc:407] Connection header received
0812 21:13:07.485784 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.485785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.485797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.485855 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.486455 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.486985 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.487913 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.488112 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.488166 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.490120 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.490121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.490122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.490129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.491396 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.491752 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.491754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.491754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.491810 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.492020 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.492021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.492021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.492156 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.492226 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.492531 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.492602 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.188@KRBTEST.COM" created.
I20260812 21:13:07.492940 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.485005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:35127 (local address 127.0.0.1:36752)
0812 21:13:07.485170 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.485206 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.485689 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.485922 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.485925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.486170 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.486365 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.486369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.487069 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.487070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.487833 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.487835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.488178 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.488179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.488290 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.488763 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.488771 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.490034 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.491471 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.491478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.491481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.491687 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.491862 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.491863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.491864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.491965 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.492225 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.492232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.492418 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.492515 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.492641 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260812 21:13:07.493513 20986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:07.500087 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.493775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50144)
0812 21:13:07.493816 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.493841 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.493872 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.494137 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.494139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.494290 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.494354 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.494356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.495158 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.495159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.495639 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.495641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.496077 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.496079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.496140 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.496516 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.496520 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.497609 (+ 1089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.499167 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.499168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.499168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.499316 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.499494 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.499494 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.499495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.499547 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.499788 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.499788 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.499826 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.499886 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.499983 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:13:07.500159 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.493858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.11.207.189:33073)
0812 21:13:07.494013 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.494017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.494032 (+ 15us) server_negotiation.cc:407] Connection header received
0812 21:13:07.494047 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.494048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.494059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.494112 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.494422 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.495078 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.495706 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.496011 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.496069 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.497683 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.497691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.497692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.497698 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.499092 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.499390 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.499393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.499393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.499442 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.499595 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.499596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.499597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.499728 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.499801 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.499916 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.500038 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260812 21:13:07.500715 20986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:07.501036 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.500936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59890)
0812 21:13:07.500971 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.501017 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 21:13:07.501168 20899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
I20260812 21:13:07.501564 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.501478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59900)
0812 21:13:07.501516 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.501545 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:13:07.501665 20986 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:40499: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab.
Entry for principal kudu/127.11.207.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab.
I20260812 21:13:07.507248 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:40499
--webserver_interface=127.11.207.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab
--principal=kudu/127.11.207.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:35127,127.11.207.189:33073,127.11.207.188:40499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:13:07.550897 20986 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } attempt: 1
I20260812 21:13:07.551462 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.551260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59916)
0812 21:13:07.551338 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.551393 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 21:13:07.551798 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.551718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59922)
0812 21:13:07.551753 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.551779 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:13:07.551893 20986 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:40499: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
I20260812 21:13:07.570849 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } attempt: 3
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:07.578310 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.571224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50152)
0812 21:13:07.571281 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.571294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.571339 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.571486 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.571487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.571652 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.571736 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.571739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.572724 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.572725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.573328 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.573331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.573836 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.573837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.573899 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.574359 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.574363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.575738 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.577425 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.577426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.577426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.577588 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.577775 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.577775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.577776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.577835 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.578083 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.578084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.578120 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.578165 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.578238 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:13:07.578387 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.571312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.11.207.189:33073)
0812 21:13:07.571378 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.571380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.571389 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:13:07.571417 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.571418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.571428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.571472 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.571833 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.572626 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.573417 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.573743 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.573811 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.575827 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.575829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.575829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.575838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.577338 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.577654 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.577655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.577656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.577718 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.577887 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.577887 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.577888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.578026 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.578094 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.578209 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.578288 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:13:07.578878 20864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:07.579246 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.579121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59930)
0812 21:13:07.579177 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.579215 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:13:07.579586 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.579504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59934)
0812 21:13:07.579540 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.579565 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:13:07.579810 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:40499: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
W20260812 21:13:07.593348 20996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:13:07.593595 20996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:07.593621 20996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:07.593649 20996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:07.594849 20996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:07.594892 20996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:07.594910 20996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:07.594926 20996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:07.594941 20996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:07.596549 20996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:35127,127.11.207.189:33073,127.11.207.188:40499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab
--principal=kudu/127.11.207.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:40499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.207.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:07.596808 20996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:07.597023 20996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:07.601897 21004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:07.601862 21003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:07.601878 21006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:07.601958 20996 server_base.cc:1034] running on GCE node
I20260812 21:13:07.602336 20996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:07.602600 20996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:07.603742 20996 hybrid_clock.cc:648] HybridClock initialized: now 1786569187603729 us; error 31 us; skew 500 ppm
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:07.606200 20996 init.cc:377] Logged in from keytab as kudu/127.11.207.188@KRBTEST.COM (short username kudu)
I20260812 21:13:07.606640 20996 webserver.cc:460] Webserver started at http://127.11.207.188:46819/ using document root <none> and password file <none>
I20260812 21:13:07.606827 20996 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:07.606868 20996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:07.606969 20996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:07.608031 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data/instance:
uuid: "64a02b129feb4cdbbefd6a99693c7202"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "7b56445e3dfa1363922ae9f4d16ffdd3"
server_key_iv: "ded92611988d9b11c0e35b3cae467425"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.608392 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal/instance:
uuid: "64a02b129feb4cdbbefd6a99693c7202"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "7b56445e3dfa1363922ae9f4d16ffdd3"
server_key_iv: "ded92611988d9b11c0e35b3cae467425"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.609743 20996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:13:07.610420 21018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:07.610560 20996 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:13:07.610646 20996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal
uuid: "64a02b129feb4cdbbefd6a99693c7202"
format_stamp: "Formatted at 2026-08-12 21:13:07 on dist-test-slave-lkx2"
server_key: "7b56445e3dfa1363922ae9f4d16ffdd3"
server_key_iv: "ded92611988d9b11c0e35b3cae467425"
server_key_version: "encryptionkey@0"
I20260812 21:13:07.610723 20996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:07.610965 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } attempt: 1
I20260812 21:13:07.611368 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.611216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59950)
0812 21:13:07.611281 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.611329 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 21:13:07.611666 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.611578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59956)
0812 21:13:07.611616 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.611644 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:13:07.611755 20864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:40499: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
I20260812 21:13:07.620048 20986 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } attempt: 2
I20260812 21:13:07.620517 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.620396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59968)
0812 21:13:07.620448 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.620488 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:13:07.620790 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.620722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59978)
0812 21:13:07.620753 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.620774 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 21:13:07.620867 20986 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:40499: Network error: Client connection negotiation failed: client connection to 127.11.207.188:40499: connect: Connection refused (error 111)
I20260812 21:13:07.630373 20996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:07.632170 20996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:07.632313 20996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:07.636533 20996 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.188:40499
I20260812 21:13:07.636579 21112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.188:40499 every 8 connection(s)
I20260812 21:13:07.636921 20996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
I20260812 21:13:07.637784 21113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:07.639413 21113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:07.642814 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 20996
I20260812 21:13:07.642903 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568904794702-12094-0/minicluster-data/master-2/wal/instance
I20260812 21:13:07.643106 12094 external_mini_cluster.cc:1428] Setting key 517c6e7417d03949b800c3defb45d7f9
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:07.648509 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.640138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.11.207.190:35127)
0812 21:13:07.640190 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.640193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.640403 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:13:07.640903 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.640904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.640914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.640964 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.641556 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.642023 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.643040 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.643327 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.643388 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.645704 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.645706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.645706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.645714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.647152 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.647533 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.647534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.647535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.647600 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.647833 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.647834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.647835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.647964 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.648016 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.648336 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.648443 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:13:07.648823 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.640106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:35127 (local address 127.0.0.1:36754)
0812 21:13:07.640273 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.640310 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.640814 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.641020 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.641023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.641270 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.641476 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.641480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.642104 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.642105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.642929 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.642931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.643416 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.643418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.643584 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.644133 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.644141 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.645621 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.647225 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.647231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.647233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.647463 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.647667 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.647668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.647668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.647774 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.648026 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.648032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.648194 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.648316 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.648513 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260812 21:13:07.649441 21113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:07.651669 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.643488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.11.207.188:40499)
0812 21:13:07.643672 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.643677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.643718 (+ 41us) server_negotiation.cc:407] Connection header received
0812 21:13:07.643740 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.643742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.643763 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.643863 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.644544 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.645331 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.646161 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.646513 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.646573 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.649032 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.649038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.649039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.649046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.650637 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.650966 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.650969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.650970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.651032 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.651207 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.651208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.651208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.651337 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.651472 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.651536 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.651596 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:07.656360 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.649693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33073 (local address 127.0.0.1:50162)
0812 21:13:07.649734 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.649744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.649780 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.650144 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.650145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.650264 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.650295 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.650296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.651060 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.651061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.651735 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.651738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.652188 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.652189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.652249 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.652613 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.652617 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.653904 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.655476 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.655477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.655477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.655628 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.655815 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.655816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.655816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.655859 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.656098 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.656099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.656130 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.656175 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.656238 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:13:07.656394 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.649964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.11.207.189:33073)
0812 21:13:07.650026 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.650029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.650037 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:13:07.650064 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.650065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.650075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.650134 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.650368 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.650989 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.651837 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.652118 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.652176 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.653977 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.653978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.653979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.653985 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.655396 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.655690 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.655691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.655691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.655751 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.655910 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.655911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.655911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.656045 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.656109 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.656203 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.656282 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:13:07.657155 21113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:07.659688 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.652635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.11.207.190:35127)
0812 21:13:07.652673 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.652675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.652681 (+ 6us) server_negotiation.cc:407] Connection header received
0812 21:13:07.652699 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.652700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.652708 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.652748 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.653287 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.653967 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.654715 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.654967 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.655017 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.657178 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.657179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.657179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.657186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.658638 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.659004 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.659005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.659005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.659049 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.659204 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.659204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.659204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.659314 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.659498 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.659533 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.659605 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:07.663270 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.657424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59986)
0812 21:13:07.657457 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.657466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.657501 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.657622 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.657623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.657733 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.657761 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.657762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.658583 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.658584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.659247 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.659249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.659668 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.659669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.659727 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.660047 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.660051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.661134 (+ 1083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.662441 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.662442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.662442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.662587 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.662738 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.662738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.662739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.662789 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.662982 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.662983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.663018 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.663078 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.663168 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 21:13:07.663319 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.657477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.11.207.188:40499)
0812 21:13:07.657526 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.657528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.657535 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:13:07.657549 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.657550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.657558 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.657605 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.657845 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.658510 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.659317 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.659589 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.659650 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.661203 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.661204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.661204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.661210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.662370 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.662642 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.662643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.662643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.662689 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.662837 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.662838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.662838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.662933 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.662983 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.663102 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.663213 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":35}
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:07.664932 21113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: Bootstrap starting.
I20260812 21:13:07.665786 21113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:07.666103 21113 log.cc:824] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:07.666851 21113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: No bootstrap required, opened a new log
I20260812 21:13:07.666913 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.660507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.11.207.189:33073)
0812 21:13:07.660549 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.660551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.660573 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:13:07.660592 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.660593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.660601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.660641 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.661090 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.661529 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.662149 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.662367 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.662420 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.664569 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.664570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.664570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.664576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.665814 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.666180 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.666181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.666181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.666238 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.666439 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.666440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.666441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.666551 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.666723 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.666759 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.666828 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:13:07.668279 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:13:07.668685 21113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.668818 21113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a02b129feb4cdbbefd6a99693c7202, State: Initialized, Role: FOLLOWER
I20260812 21:13:07.668884 21113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.669274 21113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:07.669214 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.669348 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.669394 20803 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cd7f17a47b294b29842e29fe0526153f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760
W20260812 21:13:07.670958 21137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:07.671015 21137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:07.671260 20996 master_runner.cc:186] Error getting master registration for 127.11.207.190:35127: OK, instance_id { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" instance_seqno: 1786569187339681 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:13:07.674124 20996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:13:07.686916 20864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } attempt: 2
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:07.694862 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.687384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.11.207.188:40499)
0812 21:13:07.687474 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.687477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.687508 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:13:07.687536 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.687537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.687546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.687598 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.687957 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.688721 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.689718 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.690019 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.690076 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.692212 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.692214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.692214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.692222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.693765 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.694092 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.694094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.694094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.694151 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.694324 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.694324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.694325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.694484 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.694543 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.694688 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.694767 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:13:07.694869 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.687314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:59996)
0812 21:13:07.687383 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.687396 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.687457 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.687620 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.687621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.687828 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.687885 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.687887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.688809 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.688810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.689641 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.689644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.690083 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.690084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.690143 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.690553 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.690558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.692121 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.693853 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.693854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.693855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.694026 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.694206 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.694207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.694207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.694268 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.694547 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.694548 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.694599 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.694659 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.694765 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 21:13:07.696728 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f: Bootstrap starting.
I20260812 21:13:07.697530 20864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:07.697858 20864 log.cc:824] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:07.698761 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f: No bootstrap required, opened a new log
I20260812 21:13:07.700152 20864 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.700304 20864 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7f17a47b294b29842e29fe0526153f, State: Initialized, Role: FOLLOWER
I20260812 21:13:07.700421 20864 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.700743 20864 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:07.700775 20803 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 21:13:07.700837 20803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:07.701550 20803 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.701802 20803 leader_election.cc:288] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [CANDIDATE]: Term 1 election: Requested vote from peers d51623df83d442568d7dbcf059e648da (127.11.207.189:33073), 64a02b129feb4cdbbefd6a99693c7202 (127.11.207.188:40499)
I20260812 21:13:07.700713 21142 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.702026 21142 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.702970 21070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd7f17a47b294b29842e29fe0526153f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "64a02b129feb4cdbbefd6a99693c7202"
I20260812 21:13:07.703097 21070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:07.704054 21070 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd7f17a47b294b29842e29fe0526153f in term 1.
W20260812 21:13:07.704281 21161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:07.704365 21161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:07.704236 20736 master_runner.cc:186] Error getting master registration for 127.11.207.189:33073: OK, instance_id { permanent_uuid: "d51623df83d442568d7dbcf059e648da" instance_seqno: 1786569187479728 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:13:07.706423 20736 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:13:07.706765 20763 leader_election.cc:302] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64a02b129feb4cdbbefd6a99693c7202, cd7f17a47b294b29842e29fe0526153f; no voters:
I20260812 21:13:07.706903 21142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:13:07.707062 21142 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [term 1 LEADER]: Becoming Leader. State: Replica: cd7f17a47b294b29842e29fe0526153f, State: Running, Role: LEADER
I20260812 21:13:07.707147 21142 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.707558 21142 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7f17a47b294b29842e29fe0526153f. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.707659 21142 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:07.711992 20986 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } attempt: 3
I20260812 21:13:07.713032 21186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:13:07.713304 21186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 21:13:07 dist-test-slave-lkx2 krb5kdc[20703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569187, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:07.721866 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.703512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.11.207.190:35127)
0812 21:13:07.703545 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.703547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.706489 (+ 2942us) server_negotiation.cc:407] Connection header received
0812 21:13:07.706510 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.706511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.706520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.706574 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.709978 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.710693 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.712606 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.712861 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.712904 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.716102 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.716104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.716104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.716112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.717931 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.719219 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.719220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.719220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.719547 (+ 327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.720181 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.720182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.720182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.720420 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.720522 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.721684 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.721773 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:13:07.722751 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.703260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.11.207.188:40499)
0812 21:13:07.703307 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.703309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.706604 (+ 3295us) server_negotiation.cc:407] Connection header received
0812 21:13:07.706625 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.706626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.706635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.706681 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.710115 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.710815 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.713420 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.713682 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.713730 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.718345 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.718347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.718347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.718355 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.720530 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.720977 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.720978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.720979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.721079 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.721274 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.721275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.721276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.721412 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.721485 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.722580 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.722658 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:13:07.722899 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.712279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:40499 (local address 127.0.0.1:60020)
0812 21:13:07.712363 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:07.712373 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:07.712407 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:07.712716 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:07.712717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:07.712904 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:07.712944 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.712945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.714463 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.714464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.715254 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:07.715257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.716059 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.716060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:07.716129 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.716539 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:07.716544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:07.718312 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:07.721658 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.721659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.721660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.721879 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.722159 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:07.722160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:07.722160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:07.722228 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:07.722587 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:07.722600 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:07.722644 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:13:07.722690 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:13:07.722788 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 21:13:07.723464 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.712365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.11.207.188:40499)
0812 21:13:07.712513 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.712515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.712545 (+ 30us) server_negotiation.cc:407] Connection header received
0812 21:13:07.712565 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.712566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.712575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.713025 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.713085 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.714264 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.715404 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.715934 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.716289 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.718431 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.718433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.718433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.718440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.721541 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.721969 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.721970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.721970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.722073 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.722302 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.722303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.722303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.722494 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.723094 (+ 600us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.723207 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.723331 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260812 21:13:07.724530 21069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [term 1 FOLLOWER]: Refusing update from remote peer cd7f17a47b294b29842e29fe0526153f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:13:07.724877 21179 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:13:07.726007 20986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: Bootstrap starting.
I20260812 21:13:07.726327 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:07.703530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.11.207.189:33073)
0812 21:13:07.703574 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.703577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.704711 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 21:13:07.704805 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.704807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.704818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.704847 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.705625 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.706311 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.707312 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:07.708149 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:07.708198 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:07.723223 (+ 15025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:07.723225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:07.723225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:07.723233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:07.724789 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.725300 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.725301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.725302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.725370 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:07.725648 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:07.725649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:07.725650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:07.725789 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:07.725864 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:07.726178 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 21:13:07.726253 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:13:07.726830 20986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:07.727147 20986 log.cc:824] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:07.727236 21179 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.727365 21179 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:07.727424 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7f17a47b294b29842e29fe0526153f. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.727516 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.727690 21190 mvcc.cc:205] Tried to move back new op lower bound from 7317787392915963904 to 7317787392849240064. Current Snapshot: MvccSnapshot[applied={T|T < 7317787392915963904}]
I20260812 21:13:07.727820 21119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.727907 21119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.727994 21191 mvcc.cc:205] Tried to move back new op lower bound from 7317787392915963904 to 7317787392849240064. Current Snapshot: MvccSnapshot[applied={T|T < 7317787392915963904}]
I20260812 21:13:07.732882 21186 catalog_manager.cc:1347] Generated new cluster ID: 404b2edbd0c34a5489c8b4c75a46476e
I20260812 21:13:07.732939 21186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:13:07.733177 20986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: No bootstrap required, opened a new log
I20260812 21:13:07.735347 20986 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.735497 20986 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51623df83d442568d7dbcf059e648da, State: Initialized, Role: FOLLOWER
I20260812 21:13:07.735582 20986 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } }
I20260812 21:13:07.735917 20986 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:07.737243 20939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:07.738180 20939 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [term 1 FOLLOWER]: Refusing update from remote peer cd7f17a47b294b29842e29fe0526153f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:13:07.738476 21142 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:13:07.738754 20940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd7f17a47b294b29842e29fe0526153f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d51623df83d442568d7dbcf059e648da"
I20260812 21:13:07.739540 20940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd7f17a47b294b29842e29fe0526153f in term 1.
I20260812 21:13:07.741214 21199 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.741326 21199 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.741390 21199 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7f17a47b294b29842e29fe0526153f. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.741441 21199 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.741492 21186 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:13:07.742194 21186 catalog_manager.cc:1504] Loading token signing keys...
W20260812 21:13:07.746058 21230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:07.746124 21230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:07.746229 21198 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.746296 21198 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:07.748749 21204 mvcc.cc:205] Tried to move back new op lower bound from 7317787392915963904 to 7317787392849240064. Current Snapshot: MvccSnapshot[applied={T|T < 7317787392915963904}]
I20260812 21:13:07.752578 21179 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd7f17a47b294b29842e29fe0526153f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd7f17a47b294b29842e29fe0526153f" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 35127 } } peers { permanent_uuid: "d51623df83d442568d7dbcf059e648da" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33073 } } peers { permanent_uuid: "64a02b129feb4cdbbefd6a99693c7202" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 40499 } } }
I20260812 21:13:07.752689 21179 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:07.755144 21186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd7f17a47b294b29842e29fe0526153f: Generated new TSK 0
I20260812 21:13:07.755334 21186 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:13:07.783830 20867 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.782158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.11.207.190:35127)
0812 21:13:07.782204 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.782207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.783134 (+ 927us) server_negotiation.cc:407] Connection header received
0812 21:13:07.783245 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.783247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.783258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.783288 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.783741 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36776: BlockingRecv error: recv got EOF from 127.0.0.1:36776 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:13:07.784607 21188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.782521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.11.207.188:40499)
0812 21:13:07.782565 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.782567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.784087 (+ 1520us) server_negotiation.cc:407] Connection header received
0812 21:13:07.784181 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.784182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.784193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.784220 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.784544 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60034: BlockingRecv error: recv got EOF from 127.0.0.1:60034 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 21:13:07.785306 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.782865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.11.207.189:33073)
0812 21:13:07.782903 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.782905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.784834 (+ 1929us) server_negotiation.cc:407] Connection header received
0812 21:13:07.784926 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.784928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.784937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.784964 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.785245 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50178: BlockingRecv error: recv got EOF from 127.0.0.1:50178 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:13:07.787689 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.786901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50194 (local address 127.11.207.189:33073)
0812 21:13:07.786939 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.786940 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.787202 (+ 262us) server_negotiation.cc:407] Connection header received
0812 21:13:07.787301 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.787301 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.787321 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.787346 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.787643 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50194: BlockingRecv error: recv got EOF from 127.0.0.1:50194 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 21:13:07.790663 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.789923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50204 (local address 127.11.207.189:33073)
0812 21:13:07.789956 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.789958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.790177 (+ 219us) server_negotiation.cc:407] Connection header received
0812 21:13:07.790267 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.790268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.790279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.790319 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.790618 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50204: BlockingRecv error: recv got EOF from 127.0.0.1:50204 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:13:07.794533 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.793765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.11.207.189:33073)
0812 21:13:07.793797 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.793799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.794062 (+ 263us) server_negotiation.cc:407] Connection header received
0812 21:13:07.794150 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.794150 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.794161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.794186 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.794490 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50206: BlockingRecv error: recv got EOF from 127.0.0.1:50206 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 21:13:07.800479 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.799687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.11.207.189:33073)
0812 21:13:07.799723 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.799725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.799935 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:13:07.800030 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.800031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.800042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.800069 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.800430 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50212: BlockingRecv error: recv got EOF from 127.0.0.1:50212 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:13:07.810626 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.809763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.11.207.189:33073)
0812 21:13:07.809808 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.809810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.810048 (+ 238us) server_negotiation.cc:407] Connection header received
0812 21:13:07.810147 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.810148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.810159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.810187 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.810572 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50224: BlockingRecv error: recv got EOF from 127.0.0.1:50224 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:13:07.829038 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.828052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.11.207.189:33073)
0812 21:13:07.828096 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.828098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.828413 (+ 315us) server_negotiation.cc:407] Connection header received
0812 21:13:07.828524 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.828525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.828538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.828567 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.828977 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50228: BlockingRecv error: recv got EOF from 127.0.0.1:50228 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 21:13:07.863560 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.862603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.11.207.189:33073)
0812 21:13:07.862650 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.862652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.862926 (+ 274us) server_negotiation.cc:407] Connection header received
0812 21:13:07.863029 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.863030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.863043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.863070 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.863497 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50238: BlockingRecv error: recv got EOF from 127.0.0.1:50238 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:13:07.929703 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:07.928992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50246 (local address 127.11.207.189:33073)
0812 21:13:07.929051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:07.929054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:07.929164 (+ 110us) server_negotiation.cc:407] Connection header received
0812 21:13:07.929229 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:07.929230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:07.929240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:07.929303 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:07.929638 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50246: BlockingRecv error: recv got EOF from 127.0.0.1:50246 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 21:13:08.059537 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:08.058881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.11.207.189:33073)
0812 21:13:08.058971 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.058974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.059037 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:13:08.059093 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.059094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.059104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.059156 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:08.059474 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50248: BlockingRecv error: recv got EOF from 127.0.0.1:50248 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 21:13:08.317298 20992 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:08.316751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.11.207.189:33073)
0812 21:13:08.316808 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.316810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.316822 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:13:08.316887 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.316888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.316898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.316949 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:08.317248 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50264: BlockingRecv error: recv got EOF from 127.0.0.1:50264 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:13:08.671486 21137 catalog_manager.cc:1259] Loaded cluster ID: 404b2edbd0c34a5489c8b4c75a46476e
I20260812 21:13:08.671568 21137 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: loading cluster ID for follower catalog manager: success
I20260812 21:13:08.672430 21137 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: acquiring CA information for follower catalog manager: success
I20260812 21:13:08.672729 21137 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64a02b129feb4cdbbefd6a99693c7202: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:13:08.746584 21230 catalog_manager.cc:1259] Loaded cluster ID: 404b2edbd0c34a5489c8b4c75a46476e
I20260812 21:13:08.746663 21230 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: loading cluster ID for follower catalog manager: success
I20260812 21:13:08.747496 21230 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: acquiring CA information for follower catalog manager: success
I20260812 21:13:08.747797 21230 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d51623df83d442568d7dbcf059e648da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:13:08.833155 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:08.830574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.11.207.189:33073)
0812 21:13:08.830715 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.830718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.830729 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:13:08.830785 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.830786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.830864 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.830937 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:08.831089 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.831602 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.832380 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.832596 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.832655 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:08.832807 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:08.832924 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:08.832994 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:08.833039 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 21:13:08.833062 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260812 21:13:08.837239 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:08.834865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.11.207.188:40499)
0812 21:13:08.835000 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.835002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.835010 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:13:08.835049 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.835049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.835087 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.835158 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:08.835247 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.835722 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.836456 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.836662 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.836768 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:08.836941 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:08.837058 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:08.837116 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:08.837133 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 21:13:08.837153 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260812 21:13:08.841850 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:08.839151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.11.207.188:40499)
0812 21:13:08.839189 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.839190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.839310 (+ 120us) server_negotiation.cc:407] Connection header received
0812 21:13:08.839409 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.839410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.839420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.839447 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:08.839588 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.840051 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.841104 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.841378 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.841445 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:08.841618 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:08.841654 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:08.841700 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:08.841748 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 21:13:08.841770 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:13:08.842501 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:08.838931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.11.207.190:35127)
0812 21:13:08.839035 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.839037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.839046 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:13:08.839099 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.839100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.839155 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.839255 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:08.839406 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.840187 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.841182 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.841476 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.841530 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:08.841717 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:08.841833 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:08.841900 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:08.842386 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 21:13:08.842416 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
I20260812 21:13:08.842785 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:08.838896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.11.207.189:33073)
0812 21:13:08.838936 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:08.838938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:08.839034 (+ 96us) server_negotiation.cc:407] Connection header received
0812 21:13:08.839201 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:08.839202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:08.839213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:08.839260 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:08.839467 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.840185 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.841993 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:08.842274 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:08.842349 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:08.842512 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:08.842548 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:08.842598 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:08.842690 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 21:13:08.842719 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:13:08.844130 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20736
I20260812 21:13:08.864100 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20875
I20260812 21:13:08.873029 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 20996
2026-08-12T21:13:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2978 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:13:08 dist-test-slave-lkx2 krb5kdc[21386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:13:08 dist-test-slave-lkx2 krb5kdc[21386](info): set up 2 sockets
Aug 12 21:13:08 dist-test-slave-lkx2 krb5kdc[21386](info): commencing operation
krb5kdc: starting...
W20260812 21:13:10.094303 12094 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/test-user.keytab.
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.207.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
Entry for principal kudu/127.11.207.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab.
I20260812 21:13:10.215214 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34001
--webserver_interface=127.11.207.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34001,127.11.207.189:33263,127.11.207.188:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:13:10.299687 21410 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:13:10.299926 21410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:10.299945 21410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:10.299966 21410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:10.301203 21410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:10.301252 21410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:10.301268 21410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:10.301282 21410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:10.301302 21410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:10.302817 21410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34001,127.11.207.189:33263,127.11.207.188:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.190.keytab
--principal=kudu/127.11.207.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.190:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.207.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:10.303032 21410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:10.303224 21410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:10.308060 21418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:10.308060 21419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:10.308061 21421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:10.308430 21410 server_base.cc:1034] running on GCE node
I20260812 21:13:10.308640 21410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:10.308943 21410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:10.310102 21410 hybrid_clock.cc:648] HybridClock initialized: now 1786569190310078 us; error 34 us; skew 500 ppm
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:10.317340 21410 init.cc:377] Logged in from keytab as kudu/127.11.207.190@KRBTEST.COM (short username kudu)
I20260812 21:13:10.317849 21410 webserver.cc:460] Webserver started at http://127.11.207.190:39589/ using document root <none> and password file <none>
I20260812 21:13:10.318089 21410 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:10.318138 21410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:10.318271 21410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:10.319438 21410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data/instance:
uuid: "9d40d4d4aa674e0d9b699560cbafd97a"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "65a56249141f33b41bc19b16c32dfcc1"
server_key_iv: "4004586bb5374a550f0f3555efc92900"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.319799 21410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal/instance:
uuid: "9d40d4d4aa674e0d9b699560cbafd97a"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "65a56249141f33b41bc19b16c32dfcc1"
server_key_iv: "4004586bb5374a550f0f3555efc92900"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.321184 21410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:13:10.321875 21433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:10.322027 21410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:13:10.322104 21410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal
uuid: "9d40d4d4aa674e0d9b699560cbafd97a"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "65a56249141f33b41bc19b16c32dfcc1"
server_key_iv: "4004586bb5374a550f0f3555efc92900"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.322170 21410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:10.351321 21410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:10.353587 21410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:10.353737 21410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:10.357774 21410 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.190:34001
I20260812 21:13:10.357807 21529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.190:34001 every 8 connection(s)
I20260812 21:13:10.358162 21410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/data/info.pb
I20260812 21:13:10.359025 21530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:10.360646 21530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:10.361109 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21410
I20260812 21:13:10.361200 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-0/wal/instance
I20260812 21:13:10.361423 12094 external_mini_cluster.cc:1428] Setting key 4f8f48633e35199e31ebb13ce907d6eb
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.207.189@KRBTEST.COM; defaulting to no policy
I20260812 21:13:10.371483 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.361386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34001 (local address 127.0.0.1:41598)
0812 21:13:10.361654 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.361695 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.362297 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.362571 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.362574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.362942 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.363173 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.363181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.364640 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.364641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.365653 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.365657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.366128 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.366129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.366213 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.366695 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.366706 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.368335 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.369948 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.369956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.369959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.370206 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.370456 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.370456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.370457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.370546 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.370878 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.370884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.370951 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.371033 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.371281 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":113,"thread_start_us":91,"threads_started":1}
I20260812 21:13:10.371485 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.361389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.11.207.190:34001)
0812 21:13:10.361631 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.361640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.361800 (+ 160us) server_negotiation.cc:407] Connection header received
0812 21:13:10.362439 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.362441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.362458 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.362543 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.363275 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.364546 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.365729 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.366038 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.366117 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.368440 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.368448 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.368448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.368455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.369865 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.370282 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.370284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.370284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.370394 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.370602 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.370603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.370603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.370813 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.370887 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.371085 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.371228 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260812 21:13:10.372298 21530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:10.372663 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.372557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40418)
0812 21:13:10.372598 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.372642 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 21:13:10.372817 21439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.189@KRBTEST.COM" created.
I20260812 21:13:10.373229 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.373138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40428)
0812 21:13:10.373179 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.373204 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 21:13:10.373349 21530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:33263: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab.
Entry for principal kudu/127.11.207.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab.
I20260812 21:13:10.387414 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:33263
--webserver_interface=127.11.207.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab
--principal=kudu/127.11.207.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34001,127.11.207.189:33263,127.11.207.188:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:13:10.422582 21530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } attempt: 1
I20260812 21:13:10.423105 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.422974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40440)
0812 21:13:10.423028 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.423072 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:13:10.423504 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.423410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40446)
0812 21:13:10.423455 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.423482 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 21:13:10.423611 21530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:33263: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
W20260812 21:13:10.471200 21542 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:13:10.471427 21542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:10.471453 21542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:10.471479 21542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:10.472795 21542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:10.472839 21542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:10.472858 21542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:10.472873 21542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:10.472889 21542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:10.474440 21542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34001,127.11.207.189:33263,127.11.207.188:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.189.keytab
--principal=kudu/127.11.207.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.189:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.207.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:10.474704 21542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:10.474920 21542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:10.479645 21550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:10.479676 21551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:10.479686 21557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:10.479717 21542 server_base.cc:1034] running on GCE node
I20260812 21:13:10.480119 21542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:10.480443 21542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:10.481581 21542 hybrid_clock.cc:648] HybridClock initialized: now 1786569190481569 us; error 30 us; skew 500 ppm
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:10.484061 21542 init.cc:377] Logged in from keytab as kudu/127.11.207.189@KRBTEST.COM (short username kudu)
I20260812 21:13:10.484535 21542 webserver.cc:460] Webserver started at http://127.11.207.189:39647/ using document root <none> and password file <none>
I20260812 21:13:10.484736 21542 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:10.484794 21542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:10.484926 21542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:10.485966 21542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data/instance:
uuid: "83dd33aa6aef42f6ab83e059fb163707"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "5925d19322373d0015b7463ab1741d64"
server_key_iv: "2bd10b9c5617d1f156538d914c108ada"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.486300 21542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal/instance:
uuid: "83dd33aa6aef42f6ab83e059fb163707"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "5925d19322373d0015b7463ab1741d64"
server_key_iv: "2bd10b9c5617d1f156538d914c108ada"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.487524 21542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:13:10.488219 21567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:10.488404 21542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:13:10.488497 21542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal
uuid: "83dd33aa6aef42f6ab83e059fb163707"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "5925d19322373d0015b7463ab1741d64"
server_key_iv: "2bd10b9c5617d1f156538d914c108ada"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.488564 21542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:10.491750 21530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } attempt: 2
I20260812 21:13:10.492153 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.492019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40450)
0812 21:13:10.492075 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.492117 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:13:10.492491 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.492418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40456)
0812 21:13:10.492453 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.492474 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:13:10.492623 21530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.189:33263: Network error: Client connection negotiation failed: client connection to 127.11.207.189:33263: connect: Connection refused (error 111)
I20260812 21:13:10.505847 21542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:10.507956 21542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:10.508085 21542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:10.512039 21542 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.189:33263
I20260812 21:13:10.512071 21668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.189:33263 every 8 connection(s)
I20260812 21:13:10.512452 21542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/data/info.pb
I20260812 21:13:10.512969 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21542
I20260812 21:13:10.513085 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-1/wal/instance
I20260812 21:13:10.513302 12094 external_mini_cluster.cc:1428] Setting key 730ffbb9081d172a3f9d6c109b5e374e
I20260812 21:13:10.513520 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:10.515482 21669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.207.188@KRBTEST.COM; defaulting to no policy
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.207.188@KRBTEST.COM" created.
I20260812 21:13:10.524569 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.516182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.11.207.190:34001)
0812 21:13:10.516233 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.516235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.516477 (+ 242us) server_negotiation.cc:407] Connection header received
0812 21:13:10.516992 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.516993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.517009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.517057 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.517670 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.518181 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.519333 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.519553 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.519612 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.521842 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.521844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.521844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.521851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.523049 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.523509 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.523510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.523510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.523567 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.523815 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.523816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.523816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.523923 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.523990 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.524371 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.524463 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:13:10.524753 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.516146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34001 (local address 127.0.0.1:41606)
0812 21:13:10.516327 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.516391 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.516896 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.517138 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.517142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.517387 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.517582 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.517587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.518261 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.518262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.519216 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.519219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.519629 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.519630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.519752 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.520253 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.520261 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.521728 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.523148 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.523155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.523157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.523440 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.523615 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.523615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.523615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.523744 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.524013 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.524021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.524188 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.524295 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.524471 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260812 21:13:10.525292 21669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:10.532385 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.525540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40464)
0812 21:13:10.525723 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.525735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.525779 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.525949 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.525950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.526090 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.526148 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.526149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.526979 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.526980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.527471 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.527472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.528084 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.528084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.528134 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.528548 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.528552 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.529661 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.531452 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.531453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.531453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.531606 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.531797 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.531797 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.531797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.531853 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.532136 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.532136 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.532168 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.532210 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.532283 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260812 21:13:10.532503 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.525596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40464 (local address 127.11.207.189:33263)
0812 21:13:10.525790 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.525795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.525823 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:13:10.525849 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.525851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.525866 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.525940 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.526208 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.526890 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.527547 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.528012 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.528078 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.529742 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.529751 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.529751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.529758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.531371 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.531677 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.531680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.531680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.531737 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.531903 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.531904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.531905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.532078 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.532161 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.532243 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.532371 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260812 21:13:10.533147 21669 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:10.533497 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.533375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39070)
0812 21:13:10.533424 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.533473 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 21:13:10.533622 21573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
I20260812 21:13:10.534017 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.533907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39082)
0812 21:13:10.533946 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.533999 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:13:10.534107 21669 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:38545: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.207.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab.
Entry for principal kudu/127.11.207.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab.
I20260812 21:13:10.539412 12094 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwFLJxB/build/release/bin/kudu
/tmp/dist-test-taskwFLJxB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:38545
--webserver_interface=127.11.207.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab
--principal=kudu/127.11.207.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.207.148:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.207.190:34001,127.11.207.189:33263,127.11.207.188:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:13:10.583305 21669 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } attempt: 1
I20260812 21:13:10.583791 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.583663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39094)
0812 21:13:10.583716 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.583752 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 21:13:10.583768 21530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } attempt: 3
I20260812 21:13:10.584218 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.584089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39098)
0812 21:13:10.584141 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.584184 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 21:13:10.584414 21669 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:38545: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:10.590142 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.584045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40480)
0812 21:13:10.584094 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.584104 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.584150 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.584256 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.584257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.584473 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.584569 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.584572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.585213 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.585214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.585758 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.585760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.586125 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.586127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.586201 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.586649 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.586654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.587831 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.589238 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.589239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.589239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.589401 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.589567 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.589568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.589568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.589625 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.589846 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.589846 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.589893 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.589952 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.590046 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:13:10.590138 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.584121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.11.207.189:33263)
0812 21:13:10.584153 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.584155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.584164 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:13:10.584194 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.584195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.584206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.584251 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.584647 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.585136 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.585839 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.586063 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.586120 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.587910 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.587912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.587912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.587920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.589163 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.589461 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.589463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.589463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.589515 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.589674 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.589674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.589675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.589782 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.589839 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.589981 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.590050 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:13:10.590550 21530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:10.590904 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.590763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39110)
0812 21:13:10.590828 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.590871 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 21:13:10.591207 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.591118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39116)
0812 21:13:10.591160 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.591187 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 21:13:10.591315 21530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:38545: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
I20260812 21:13:10.622471 21530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } attempt: 1
I20260812 21:13:10.622959 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.622794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39120)
0812 21:13:10.622865 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.622919 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 21:13:10.623354 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.623224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39136)
0812 21:13:10.623298 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.623331 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 21:13:10.623462 21530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:38545: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
W20260812 21:13:10.623669 21681 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:13:10.623870 21681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:13:10.623904 21681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:13:10.623939 21681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:13:10.625227 21681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:13:10.625272 21681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:13:10.625291 21681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:13:10.625308 21681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:13:10.625331 21681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:13:10.626873 21681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.207.148:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.207.190:34001,127.11.207.189:33263,127.11.207.188:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/krb5kdc/kudu_127.11.207.188.keytab
--principal=kudu/127.11.207.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.207.188:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.207.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:13:10.627113 21681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:13:10.627326 21681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:13:10.632102 21688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:10.632092 21697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:13:10.632092 21689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:13:10.632328 21681 server_base.cc:1034] running on GCE node
I20260812 21:13:10.632649 21681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:13:10.632942 21681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:13:10.634105 21681 hybrid_clock.cc:648] HybridClock initialized: now 1786569190634086 us; error 29 us; skew 500 ppm
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:13:10.636567 21681 init.cc:377] Logged in from keytab as kudu/127.11.207.188@KRBTEST.COM (short username kudu)
I20260812 21:13:10.637025 21681 webserver.cc:460] Webserver started at http://127.11.207.188:33429/ using document root <none> and password file <none>
I20260812 21:13:10.637262 21681 fs_manager.cc:359] Metadata directory not provided
I20260812 21:13:10.637310 21681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:13:10.637441 21681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:13:10.638507 21681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data/instance:
uuid: "dffaa62c988349ee95e1a76553179067"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "1bb6b7b866c96afda90fd29062f75546"
server_key_iv: "ef2f3650ee53d6976cb4a57a47254419"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.638846 21681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal/instance:
uuid: "dffaa62c988349ee95e1a76553179067"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "1bb6b7b866c96afda90fd29062f75546"
server_key_iv: "ef2f3650ee53d6976cb4a57a47254419"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.640125 21681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:13:10.640883 21714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:10.641034 21681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:13:10.641121 21681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data,/tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal
uuid: "dffaa62c988349ee95e1a76553179067"
format_stamp: "Formatted at 2026-08-12 21:13:10 on dist-test-slave-lkx2"
server_key: "1bb6b7b866c96afda90fd29062f75546"
server_key_iv: "ef2f3650ee53d6976cb4a57a47254419"
server_key_version: "encryptionkey@0"
I20260812 21:13:10.641191 21681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:13:10.652562 21669 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } attempt: 2
I20260812 21:13:10.653033 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.652904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39152)
0812 21:13:10.652963 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.652998 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:13:10.653434 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.653334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39162)
0812 21:13:10.653378 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.653414 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 21:13:10.653534 21669 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.207.188:38545: Network error: Client connection negotiation failed: client connection to 127.11.207.188:38545: connect: Connection refused (error 111)
I20260812 21:13:10.657065 21681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:13:10.658954 21681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:13:10.659070 21681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:13:10.663097 21681 rpc_server.cc:307] RPC server started. Bound to: 127.11.207.188:38545
I20260812 21:13:10.663138 21820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.207.188:38545 every 8 connection(s)
I20260812 21:13:10.663491 21681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/data/info.pb
I20260812 21:13:10.664439 21821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:13:10.664700 12094 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwFLJxB/build/release/bin/kudu as pid 21681
I20260812 21:13:10.664809 12094 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwFLJxB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568904794702-12094-0/minicluster-data/master-2/wal/instance
I20260812 21:13:10.665055 12094 external_mini_cluster.cc:1428] Setting key 319c9d924ce340d78325f8ba48dd7f6c
I20260812 21:13:10.666371 21821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:10.675032 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.667135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.11.207.190:34001)
0812 21:13:10.667224 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.667226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.667302 (+ 76us) server_negotiation.cc:407] Connection header received
0812 21:13:10.667366 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.667367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.667376 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.667423 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.668002 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.668496 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.669655 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.669965 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.670028 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.672080 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.672082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.672083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.672091 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.673613 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.674058 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.674059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.674060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.674122 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.674354 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.674355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.674356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.674492 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.674567 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.674843 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.674934 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 21:13:10.675196 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.667036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.190:34001 (local address 127.0.0.1:41616)
0812 21:13:10.667208 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.667247 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.667308 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.667451 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.667454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.667793 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.667923 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.667929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.668569 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.668570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.669564 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.669567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.670049 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.670050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.670119 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.670489 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.670499 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.671993 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.673691 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.673700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.673702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.673988 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.674181 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.674182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.674182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.674282 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.674557 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.674560 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.674688 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.674813 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.674959 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260812 21:13:10.675592 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.665583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39172 (local address 127.11.207.188:38545)
0812 21:13:10.665765 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.665773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.665799 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:13:10.666215 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.666221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.666273 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.666391 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.667158 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.668386 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.669428 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.669815 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.669919 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.672595 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.672605 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.672605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.672613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.674395 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.674736 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.674739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.674739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.674806 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.675002 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.675003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.675003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.675174 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.675359 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.675411 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.675492 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260812 21:13:10.675990 21821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.190@KRBTEST.COM
I20260812 21:13:10.682688 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.676239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.189:33263 (local address 127.0.0.1:40496)
0812 21:13:10.676288 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.676300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.676372 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.676511 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.676512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.676664 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.676730 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.676733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.677391 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.677392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.677913 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.677915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.678260 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.678260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.678333 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.678716 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.678720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.679907 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.681663 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.681664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.681664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.681845 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.682073 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.682073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.682074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.682137 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.682402 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.682403 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.682444 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.682498 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.682592 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:13:10.682688 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.676308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.11.207.189:33263)
0812 21:13:10.676394 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.676396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.676406 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:13:10.676433 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.676434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.676444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.676497 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.676815 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.677307 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.677993 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.678197 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.678246 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.679997 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.679999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.680000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.680009 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.681574 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.681921 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.681922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.681922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.681986 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.682203 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.682203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.682204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.682337 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.682395 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.682529 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.682598 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:13:10.683066 21821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:13:10.683502 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.676689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.11.207.190:34001)
0812 21:13:10.676726 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.676728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.676733 (+ 5us) server_negotiation.cc:407] Connection header received
0812 21:13:10.676747 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.676748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.676754 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.676796 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.677261 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.677737 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.678503 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.678763 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.678813 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.680945 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.680946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.680946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.680953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.682198 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.682498 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.682500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.682500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.682557 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.682777 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.682778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.682779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.682920 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.683092 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.683265 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.683415 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.188@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.207.189@KRBTEST.COM
I20260812 21:13:10.689616 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.683295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39178)
0812 21:13:10.683340 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.683349 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.683381 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.683477 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.683477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.683621 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.683651 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.683653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.684399 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.684400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.685001 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.685003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.685323 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.685324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.685375 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.685734 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.685739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.687052 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.688670 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.688672 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.688672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.688862 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.689057 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.689058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.689058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.689121 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.689382 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.689383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.689420 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.689472 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.689548 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:13:10.689731 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.683348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.11.207.188:38545)
0812 21:13:10.683383 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.683384 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.683391 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:13:10.683425 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.683426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.683433 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.683476 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.683716 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.684313 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.685073 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.685266 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.685314 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.687156 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.687158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.687159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.687165 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.688598 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.688933 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.688935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.688935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.688994 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.689178 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.689179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.689179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.689315 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.689387 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.689515 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.689619 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":21}
I20260812 21:13:10.691146 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.684091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.11.207.189:33263)
0812 21:13:10.684126 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.684127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.684135 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:13:10.684198 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.684199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.684206 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.684240 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.684713 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.685312 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.685924 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.686227 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.686284 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.688329 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.688331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.688331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.688362 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.690016 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.690359 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.690361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.690361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.690425 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.690613 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.690613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.690614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.690728 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.690937 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.690977 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.691052 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:13:10.691179 21821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: Bootstrap starting.
I20260812 21:13:10.692304 21821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:10.692667 21821 log.cc:824] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:10.692898 12094 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:13:10.693499 21821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: No bootstrap required, opened a new log
I20260812 21:13:10.694036 21490 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9d40d4d4aa674e0d9b699560cbafd97a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41626
I20260812 21:13:10.695591 21821 raft_consensus.cc:348] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.695756 21821 raft_consensus.cc:729] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffaa62c988349ee95e1a76553179067, State: Initialized, Role: FOLLOWER
I20260812 21:13:10.695843 21821 consensus_queue.cc:260] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.696157 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.696208 21821 sys_catalog.cc:564] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:10.696260 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:13:10.698045 21850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:10.698136 21850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:10.698614 21530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } attempt: 2
I20260812 21:13:10.698333 21681 master_runner.cc:186] Error getting master registration for 127.11.207.190:34001: OK, instance_id { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" instance_seqno: 1786569190356627 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:13:10.700731 21681 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.190@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:10.706120 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.698920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.11.207.188:38545)
0812 21:13:10.698972 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.698974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.698982 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:13:10.699031 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.699032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.699043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.699086 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.699418 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.699875 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.700896 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.701081 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.701137 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.703341 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.703342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.703343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.703352 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.704905 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.705310 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.705311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.705311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.705367 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.705561 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.705561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.705562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.705727 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.705787 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.705950 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.706028 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:13:10.706120 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.698870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39184)
0812 21:13:10.698921 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.698931 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.698969 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.699124 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.699125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.699310 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.699358 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.699360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.699946 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.699947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.700817 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.700821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.701142 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.701143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.701197 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.701669 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.701675 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.703229 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.704997 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.704998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.704998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.705234 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.705426 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.705427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.705427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.705499 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.705788 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.705789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.705844 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.705914 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.706017 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:13:10.708024 21530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a: Bootstrap starting.
I20260812 21:13:10.708894 21530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:10.709218 21530 log.cc:824] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:10.710139 21530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a: No bootstrap required, opened a new log
I20260812 21:13:10.711571 21530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.711730 21530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d40d4d4aa674e0d9b699560cbafd97a, State: Initialized, Role: FOLLOWER
I20260812 21:13:10.711812 21530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.712126 21857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.712170 21530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:10.712267 21857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:10.712185 21490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 21:13:10.713533 21490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:10.714236 21490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.714499 21490 leader_election.cc:288] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [CANDIDATE]: Term 1 election: Requested vote from peers 83dd33aa6aef42f6ab83e059fb163707 (127.11.207.189:33263), dffaa62c988349ee95e1a76553179067 (127.11.207.188:38545)
I20260812 21:13:10.714993 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dffaa62c988349ee95e1a76553179067"
I20260812 21:13:10.715134 21766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:10.716136 21766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d40d4d4aa674e0d9b699560cbafd97a in term 1.
I20260812 21:13:10.716395 21439 leader_election.cc:302] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d40d4d4aa674e0d9b699560cbafd97a, dffaa62c988349ee95e1a76553179067; no voters:
I20260812 21:13:10.716533 21857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:13:10.716674 21857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [term 1 LEADER]: Becoming Leader. State: Replica: 9d40d4d4aa674e0d9b699560cbafd97a, State: Running, Role: LEADER
I20260812 21:13:10.716768 21857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
W20260812 21:13:10.716971 21885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:10.717101 21885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:10.717173 21857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d40d4d4aa674e0d9b699560cbafd97a. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.717263 21857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:10.717455 21896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:13:10.717620 21896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:13:10.716187 21410 master_runner.cc:186] Error getting master registration for 127.11.207.189:33263: OK, instance_id { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" instance_seqno: 1786569190510968 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:13:10.718492 21410 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:13:10.719676 21766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [term 1 FOLLOWER]: Refusing update from remote peer 9d40d4d4aa674e0d9b699560cbafd97a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:13:10.720023 21894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:13:10.721428 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d40d4d4aa674e0d9b699560cbafd97a. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.721524 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:10.722571 21898 mvcc.cc:205] Tried to move back new op lower bound from 7317787405185273856 to 7317787405176590336. Current Snapshot: MvccSnapshot[applied={T|T < 7317787405185273856}]
I20260812 21:13:10.722908 21857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.722983 21857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:10.723186 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.723265 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:10.723758 21903 mvcc.cc:205] Tried to move back new op lower bound from 7317787405185273856 to 7317787405176590336. Current Snapshot: MvccSnapshot[applied={T|T < 7317787405185273856}]
I20260812 21:13:10.726651 21896 catalog_manager.cc:1347] Generated new cluster ID: b8c8b1bdc32d44818e810a4adb0b47d2
I20260812 21:13:10.726699 21896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:13:10.728919 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.720622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.11.207.189:33263)
0812 21:13:10.720669 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.720672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.720894 (+ 222us) server_negotiation.cc:407] Connection header received
0812 21:13:10.721026 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.721028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.721040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.721094 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.721849 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.722609 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.723670 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.723985 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.724027 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.725651 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.725653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.725653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.725661 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.727223 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.727656 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.727657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.727657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.727718 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.728271 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.728272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.728272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.728458 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.728532 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.728741 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.728824 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:13:10.732303 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.720940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.11.207.188:38545)
0812 21:13:10.720995 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.720997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.721481 (+ 484us) server_negotiation.cc:407] Connection header received
0812 21:13:10.721655 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.721669 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.721680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.721708 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.724470 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.725212 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.726205 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.726738 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.726812 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.728915 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.728916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.728917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.728923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.730701 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.731281 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.731282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.731282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.731350 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.731618 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.731619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.731619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.731748 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.731832 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.732135 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.732214 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:13:10.735275 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.721385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.11.207.190:34001)
0812 21:13:10.721587 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.721589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.724568 (+ 2979us) server_negotiation.cc:407] Connection header received
0812 21:13:10.725957 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.725958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.725970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.726000 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.729207 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.729853 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.730804 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.731060 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.731100 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.732481 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.732482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.732483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.732491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.734056 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.734453 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.734455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.734455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.734512 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.734694 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.734694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.734695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.734817 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.734906 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.735093 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.735180 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260812 21:13:10.744692 21669 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } attempt: 3
Aug 12 21:13:10 dist-test-slave-lkx2 krb5kdc[21386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786569190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.207.189@KRBTEST.COM for kudu/127.11.207.188@KRBTEST.COM
I20260812 21:13:10.752648 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.744962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.207.188:38545 (local address 127.0.0.1:39212)
0812 21:13:10.745010 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:13:10.745020 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:13:10.745050 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:13:10.745281 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:13:10.745282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:13:10.745468 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:13:10.745521 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.745522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.746373 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.746374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.747177 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:13:10.747179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.747607 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.747608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:13:10.747666 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.748183 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:13:10.748188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:13:10.749741 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:13:10.751563 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.751564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.751565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.751773 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.751971 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:13:10.751971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:13:10.751972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:13:10.752028 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:13:10.752305 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:13:10.752306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:13:10.752372 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 21:13:10.752431 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:13:10.752530 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:13:10.753551 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:10.745018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.11.207.188:38545)
0812 21:13:10.745160 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.745162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.745168 (+ 6us) server_negotiation.cc:407] Connection header received
0812 21:13:10.745203 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.745204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.745212 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.745252 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.745605 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.746307 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.747258 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:10.747549 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:10.747596 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:10.749839 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:13:10.749840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:13:10.749841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:13:10.749849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:13:10.751456 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.751848 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.751849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.751849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.751906 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:13:10.752086 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:13:10.752087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:13:10.752087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:13:10.752224 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:13:10.752280 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:10.753359 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0812 21:13:10.753457 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 21:13:10.755070 21669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: Bootstrap starting.
I20260812 21:13:10.755817 21669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: Neither blocks nor log segments found. Creating new log.
I20260812 21:13:10.756088 21669 log.cc:824] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: Log is configured to *not* fsync() on all Append() calls
I20260812 21:13:10.757032 21669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: No bootstrap required, opened a new log
I20260812 21:13:10.758873 21669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.759002 21669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83dd33aa6aef42f6ab83e059fb163707, State: Initialized, Role: FOLLOWER
I20260812 21:13:10.759069 21669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } }
I20260812 21:13:10.759371 21669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:13:10.759466 21625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "83dd33aa6aef42f6ab83e059fb163707"
I20260812 21:13:10.759581 21625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:13:10.760447 21625 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d40d4d4aa674e0d9b699560cbafd97a in term 1.
I20260812 21:13:10.760912 21624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [term 1 FOLLOWER]: Refusing update from remote peer 9d40d4d4aa674e0d9b699560cbafd97a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:13:10.761209 21913 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [LEADER]: Connected to new peer: Peer: permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:13:10.763732 21941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.763851 21941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:10.763998 21941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d40d4d4aa674e0d9b699560cbafd97a. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.764042 21941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:13:10.764065 21941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.764096 21941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:13:10.764533 21939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:13:10.764578 21939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:13:10.766222 21894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d40d4d4aa674e0d9b699560cbafd97a" member_type: VOTER last_known_addr { host: "127.11.207.190" port: 34001 } } peers { permanent_uuid: "83dd33aa6aef42f6ab83e059fb163707" member_type: VOTER last_known_addr { host: "127.11.207.189" port: 33263 } } peers { permanent_uuid: "dffaa62c988349ee95e1a76553179067" member_type: VOTER last_known_addr { host: "127.11.207.188" port: 38545 } } }
I20260812 21:13:10.766347 21894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a [sys.catalog]: This master's current role is: LEADER
I20260812 21:13:10.767441 21896 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:13:10.768546 21896 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:13:10.778570 21896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d40d4d4aa674e0d9b699560cbafd97a: Generated new TSK 0
I20260812 21:13:10.778770 21896 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:13:10.813565 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.811804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.11.207.189:33263)
0812 21:13:10.811856 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.811859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.812829 (+ 970us) server_negotiation.cc:407] Connection header received
0812 21:13:10.812935 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.812937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.812949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.812979 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.813487 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40528: BlockingRecv error: recv got EOF from 127.0.0.1:40528 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 21:13:10.814496 21828 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.812173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.11.207.188:38545)
0812 21:13:10.812215 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.812217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.813786 (+ 1569us) server_negotiation.cc:407] Connection header received
0812 21:13:10.814021 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.814022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.814034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.814064 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.814431 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39222: BlockingRecv error: recv got EOF from 127.0.0.1:39222 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 21:13:10.815162 21533 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.812532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.11.207.190:34001)
0812 21:13:10.812573 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.812575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.813970 (+ 1395us) server_negotiation.cc:407] Connection header received
0812 21:13:10.814732 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.814734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.814745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.814775 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.815086 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41632: BlockingRecv error: recv got EOF from 127.0.0.1:41632 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:13:10.817529 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.816746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.11.207.189:33263)
0812 21:13:10.816779 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.816781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.817013 (+ 232us) server_negotiation.cc:407] Connection header received
0812 21:13:10.817122 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.817123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.817141 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.817167 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.817481 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40536: BlockingRecv error: recv got EOF from 127.0.0.1:40536 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:13:10.820454 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.819664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.11.207.189:33263)
0812 21:13:10.819697 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.819699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.819933 (+ 234us) server_negotiation.cc:407] Connection header received
0812 21:13:10.820023 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.820024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.820034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.820061 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.820411 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40540: BlockingRecv error: recv got EOF from 127.0.0.1:40540 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:13:10.824239 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.823487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.11.207.189:33263)
0812 21:13:10.823520 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.823521 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.823778 (+ 257us) server_negotiation.cc:407] Connection header received
0812 21:13:10.823869 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.823870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.823879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.823906 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.824195 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40556: BlockingRecv error: recv got EOF from 127.0.0.1:40556 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 21:13:10.830431 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.829603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.11.207.189:33263)
0812 21:13:10.829643 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.829644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.829892 (+ 248us) server_negotiation.cc:407] Connection header received
0812 21:13:10.829985 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.829986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.829996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.830023 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.830376 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40566: BlockingRecv error: recv got EOF from 127.0.0.1:40566 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:13:10.840659 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.839657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.11.207.189:33263)
0812 21:13:10.839701 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.839703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.840009 (+ 306us) server_negotiation.cc:407] Connection header received
0812 21:13:10.840110 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.840111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.840124 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.840153 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.840598 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40576: BlockingRecv error: recv got EOF from 127.0.0.1:40576 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:13:10.859066 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.858011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.11.207.189:33263)
0812 21:13:10.858059 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.858061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.858402 (+ 341us) server_negotiation.cc:407] Connection header received
0812 21:13:10.858517 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.858519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.858531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.858563 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.859005 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40592: BlockingRecv error: recv got EOF from 127.0.0.1:40592 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 21:13:10.893554 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.892526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40606 (local address 127.11.207.189:33263)
0812 21:13:10.892570 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.892572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.892897 (+ 325us) server_negotiation.cc:407] Connection header received
0812 21:13:10.893006 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.893007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.893020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.893050 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.893496 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40606: BlockingRecv error: recv got EOF from 127.0.0.1:40606 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 21:13:10.959627 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:10.958886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.11.207.189:33263)
0812 21:13:10.958971 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:10.958974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:10.959079 (+ 105us) server_negotiation.cc:407] Connection header received
0812 21:13:10.959134 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:10.959135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:10.959146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:10.959197 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:10.959560 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40620: BlockingRecv error: recv got EOF from 127.0.0.1:40620 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 21:13:11.089531 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:11.088851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.11.207.189:33263)
0812 21:13:11.088918 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.088921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.088985 (+ 64us) server_negotiation.cc:407] Connection header received
0812 21:13:11.089063 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.089064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.089077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.089124 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:11.089474 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40634: BlockingRecv error: recv got EOF from 127.0.0.1:40634 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 21:13:11.347565 21678 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:13:11.346702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.11.207.189:33263)
0812 21:13:11.346793 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.346796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.346936 (+ 140us) server_negotiation.cc:407] Connection header received
0812 21:13:11.347047 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.347049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.347063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.347120 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:13:11.347499 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40640: BlockingRecv error: recv got EOF from 127.0.0.1:40640 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 21:13:11.698603 21850 catalog_manager.cc:1259] Loaded cluster ID: b8c8b1bdc32d44818e810a4adb0b47d2
I20260812 21:13:11.698686 21850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: loading cluster ID for follower catalog manager: success
I20260812 21:13:11.699520 21850 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: acquiring CA information for follower catalog manager: success
I20260812 21:13:11.699842 21850 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dffaa62c988349ee95e1a76553179067: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:13:11.764971 21939 catalog_manager.cc:1259] Loaded cluster ID: b8c8b1bdc32d44818e810a4adb0b47d2
I20260812 21:13:11.765055 21939 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: loading cluster ID for follower catalog manager: success
I20260812 21:13:11.765861 21939 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: acquiring CA information for follower catalog manager: success
I20260812 21:13:11.766140 21939 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 83dd33aa6aef42f6ab83e059fb163707: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:13:11.863397 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:11.860781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.11.207.189:33263)
0812 21:13:11.860913 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.860915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.860967 (+ 52us) server_negotiation.cc:407] Connection header received
0812 21:13:11.861020 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.861021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.861077 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.861150 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:11.861283 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.861779 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.862555 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.862751 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.862798 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:11.863015 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:11.863136 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:11.863209 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:11.863289 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 21:13:11.863309 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260812 21:13:11.867383 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:11.865069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.11.207.188:38545)
0812 21:13:11.865204 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.865206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.865214 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:13:11.865251 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.865251 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.865290 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.865366 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:11.865447 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.865976 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.866719 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.866923 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.866979 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:11.867077 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:11.867194 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:11.867248 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:11.867288 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 21:13:11.867304 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260812 21:13:11.871649 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:11.868899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.11.207.189:33263)
0812 21:13:11.868939 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.868941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.869180 (+ 239us) server_negotiation.cc:407] Connection header received
0812 21:13:11.869275 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.869276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.869286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.869316 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:11.869502 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.869981 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.870927 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.871219 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.871280 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:11.871407 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:11.871440 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:11.871486 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:11.871551 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 21:13:11.871573 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:13:11.871845 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:11.869141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.11.207.188:38545)
0812 21:13:11.869178 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.869180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.869223 (+ 43us) server_negotiation.cc:407] Connection header received
0812 21:13:11.869286 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.869287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.869297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.869345 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:11.869539 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.870255 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.871070 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.871355 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.871389 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:11.871586 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:11.871614 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:11.871638 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:11.871754 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 21:13:11.871778 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:13:11.872102 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:13:11.868953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.11.207.190:34001)
0812 21:13:11.869135 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 21:13:11.869137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:13:11.869144 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:13:11.869213 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:13:11.869214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:13:11.869270 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:13:11.869351 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:13:11.869449 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.870230 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.871157 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:13:11.871446 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:13:11.871500 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:13:11.871697 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:13:11.871820 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:13:11.871881 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:13:11.872015 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 21:13:11.872043 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":61,"threads_started":1}
I20260812 21:13:11.873445 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21410
I20260812 21:13:11.895494 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21542
I20260812 21:13:11.907691 12094 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwFLJxB/build/release/bin/kudu with pid 21681
2026-08-12T21:13:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3035 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6013 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287108 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST